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TITLE OF THE INVENTION: 

Method and Apparatus for Compressing and Decompressing Images 



BACKGROUND OF THE INVENTION : 
5 Field of the Invention: 

Modern computers and modern computer networks enable the 
transfer of a significant amount of information between computers and 
between a computer and a storage device. When computers access local 
storage devices, such as a local hard drive or local floppy drive, significant 

10 amounts of information can be quickly accessed. However, when seeking 
to access data from a remote storage location such as over a wide area 
network (WAN), the internet, or a wireless communication channel (cellular 
phone network, etc), data transfer rates are significantly slower. 
Transferring large files, therefore, takes significant amounts of time. 

15 Additionally, storage of large files utilizes valuable and limited storage 
space. Photographic images and similar graphical images typically are 
considered to be large files, since an image conventionally requires 
information on each picture element or pixel in the image. Photographs 
and similar graphical images, therefore, typically require over one 

20 megabyte of storage space, and therefore require significant transmission 
times over slow network communications. In recent years, therefore, 
numerous protocols and standards have been developed for compressing 
photographic images to reduce the amount of storage space required to 
store photographic images, and to reduce transfer and rendering times. 

25 The compression methods essentially create mathematical or statistical 
approximations of the original image. 

Compression methods can broadly be categorized into two separate 
categories: Lossy compression methods are methods wherein there is a 
certain amount of loss of fidelity of the image; in other words, close 

30 inspection of the reproduced image would show a loss of fidelity of the 
image. Lossless compression methods are ones where the original image 
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is reproduced exactly after decoding. The present invention is directed to 
an efficient image compression method and apparatus wherein a part, or 
parts, of an image can be compressed with a higher level of fidelity in the 
reproduced image than other parts of the image, based on a selection of 
5 region-of-interests by the user or the system which is initially encoding or 
compressing the image, or the user or the system which receives and 
decodes the image data through interaction with the encoding side. 
Description of the Related Art: 

A currently popular standard for compressing images is called the 
10 JPEG or "J-peg" standard. This standard was developed by a committee 
called The Joint Photographic Experts Group, and is popularly used to 
compress still images for storage or network transmission. Recent papers 
by Said and Pearlman discuss new image coding and decoding methods 
based upon set partitioning in hierarchical trees (SPIHT). See Said and 
15 Pearlman, Image Codec Based on Set Partitioning in Hierarchical Trees. 
IEEE Transactions on Circuits and Systems for Video Technology, vol. 6. 
no. 3, June 1996, and Said and Pearlman, Image Multi-Resolution 
Representation, IEEE Transactions on Image Processing, vol. 5, no. 9. 
September 1996. The contents of these papers are hereby incorporated 
20 by reference. These references disclose computer software which, when 
loaded and running on a general purpose computer, performs a method 
and creates an apparatus which utilizes integer wavelet transforms which 
provide lossy compression by bit accuracy and lossless compression within 
a same embedded bit stream, or an apparatus which utilizes non-integer 
25 wavelet transforms which provide lossy compression by bit accuracy within 
a single embedded bit stream. An image which is initially stored as a two 
dimensional array representing a plurality of individual pixels prioritizes bits 
according to a transform coefficient for progressive image transmission. 
The most important information is selected by determining significant or 
30 insignificant elements with respect to a given threshold utilizing subset 
partitioning. The progressive transmission scheme disclosed by Said and 
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Pearlman selects the most important information to be transmitted first 
based upon the magnitude of each transform coefficient; if the transform is 
unitary, the larger the magnitude, the more information the coefficient 
conveys in the mean squared error (MSE, D mse ()) sense; 
5 D mse (p - P) = II P t = Jj,. X £ (Pu - Pi,) 2 

where (i,j) is the pixel coordinate, with p, therefore representing a pixel 
value. Two dimensional array c is coded according to c = O (p), with Q( ) 
10 being used to represent a unitary hierarchical subband transformation. 
Said and Pearlman make the assumption that each pixel coordinate and 
value is represented according to a fixed-point binary format with a 
relatively small number of bits which enables the element to be treated as 
an integer for the purposes of coding. The reconstructed image p is 
15 performed by setting a reconstruction vector c to 0, and calculating the 
image as: 

p = n- 1 (£) 

N is the number of image pixels, and the above calculation for mean 
squared-error distortion can therefore be made. Using mathematical 
20 assumptions, it is known that the mean squared-error distortion measure 
decreases by || c, , ll 2 /N. This fact enables pixel values to be ranked 
according to their binary representation, with the most significant bits 
(MSBs) being transmitted first, and also enables pixel coefficients with 
larger magnitude to be transmitted first because of a larger content of 
25 information. An algorithm is utilized by the encoder to send a value 
representing the maximum pixel value for a particular pixel coordinate, 
sorting pixel coordinates by wavelet transform coefficient values, then 
outputting a most significant bit of the various coefficients, using a number 
of sorting passes and refinement passes, to provide high quality 
30 reconstructed images utilizing a small fraction of the transmitted pixel 
coordinates. A user can set a desired rate or distortion by setting the 
number of bits to be spent in sorting passes and refinement passes. 
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gi IMMARY OF THE INVENTION: 

The invention is a method and apparatus for encoding images for 
transmission or storage where a region of interest (ROI) or certain regions 
of the image are to be emphasized and for decoding the encoded image 
5 after transmission or retrieval from storage. The encoding method includes 
selecting a region or regions of interest in digital image data, and 
specifying a priority to each region. A wavelet transform of the pixel values 
of the entire image is performed in order to obtain the transform 
coefficients of the wavelet, and the transform coefficients corresponding to 
10 each region of interest are identified. The transform coefficients for each 
region of interest are emphasized by scaling up these transform 
coefficients in such a way that more bits are allocated to these transform 
coefficients or encoding ordering of these coefficients are advanced. After 
the scaling up the transform coefficients for each region of interest, 
1 5 quantization is performed on the transform coefficients for the entire image 
in order to obtain the quantization indices. In the alternative, the 
quantization indices of the quantized transform coefficients corresponding 
to each region of interest are scaled up according to the priority assigned 
to each region of interest. After the quantization for the entire image. 
20 scaling up is performed for each region of interest. The quantization 
indices of the transform coefficients are entropy encoded based upon the 
encoding strategy such as encoding ordering or bit allocation determined 
by the scaling up for each region of interest in order to form a data bit 
stream. A bit stream header is formed, and the data bit stream is 
appended to the bit stream header. The entropy coding is performed on 
each bitfield of the binary representation of the quantization indices of the 
transform coefficients. Either bit plane coding is used, such as a binary 
arithmetic coding technique, or a zero-tree coding technique, such as 
SPIHT coding, is used. The decoding method includes separating the bit 
stream header from the data bit stream, decoding the description such as 
coordinates of the region or regions of interest, priority to each region, size 
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of the image, and the number of wavelet decomposition levels from the bit 
stream header. The wavelet transform coefficients corresponding to a 
region or regions of interest specified by the description of the region or the 
regions of interest are identified, and the data stream is entropy decoded 
5 by following the decoding ordering determined by the identified result of the 
transform coefficients corresponding to each region of interest and the 
priority assigned to each region of interest. This forms a set of subbands 
containing the quantization indices of the transform coefficients. Either the 
de-quantized transform coefficients or the quantization indices of the 
10 transform coefficients corresponding to each region of interest are scaled 
down. If scaling up and quantization are performed in this order at the 
encoder, de-quantization of the transform coefficients for the entire image 
and scaling down the quantized transform coefficients for each region of 
interest is performed in this order; if quantization and scaling up are 
15 performed in this order at the encoder, scaling down the quantization 
indices for each region of interest and de-quantization of the quantization 
indices for the entire image is performed in this order. In either case, de- 
quantization is performed on the quantization indices in order to obtain the 
quantized transform coefficients. The inverse wavelet transform is 
20 performed on the de-quantized transform coefficients in order to form the 
pixel values on the entire image. The digital image in this invention can be 
not only two dimensional digital data but also one dimensional digital data 
such as voice data, electrocardiogram data, seismic wave data. When the 
data is one dimensional, steps and means based on wavelet transform, 
25 subband, ROI coefficient identification or inverse wavelet transform which 
are applied along each dimension of the two dimensional data are applied 
only along the single dimension of the data. 
RR1FF RESCRIPT'"" OP THF. DRAWI NGS: 

For a clear understanding of the embodiments of the invention. 
30 reference should be made to the accompanying drawings, wherein: 
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Fig. 1 illustrates a method for compressing an image with an 
emphasis on a selected region of interest in an image by allocating more 
bits to the region of interest; 

Fig. 2 illustrates a method of decompressing an image which is 
encoded with an emphasis on a selected ROI; 

Figs. 3A-3F illustrate a method of ROI coefficient scaling; 

Figs. 4A and 4B illustrate an ROI coefficient scaling method wherein 
multiple regions of interest are to be emphasized with different priorities; 

Figs. 5A-5C illustrate an ROI coefficient scaling method that uses 
) some of the bit elements in the quantization indices for a ROI in order to 

emphasize the ROI; 

Figs. 6A-6C illustrate an ROI coefficient scaling method where 
multiple regions of interest are emphasized from different stages of 
encoding with different priority is disclosed; 
5 Figs. 7A-7D illustrate entropy coding of the quantization indices with 

a bit plane based coder under the encoding strategy determined by the 
ROI coefficient scaling. 

Figs. 8A-8C illustrate the case where bit plane coding is performed 
in each subband where ROI coefficient scaling is used only on the bit fields 
20 at some bit significance levels; 

Fig. 9 illustrates an identification process performed only on the 
pixels on the boundaries of the region of interest and on the pixels which 
locate one pixel inside the boundaries of the region of interest; 

Figs. 10A and 10B illustrate a wavelet transform accomplished by a 
25 low pass filter whose filter coefficients are g A (k) and a high pass filter 
whose filter coefficients are f A (k) and a down sampler which discards every 
other pixel or transform coefficient; 

Fig. 11 illustrates an encoding method for representing the input 
image with a set of subbands consisting of the transform coefficients; 
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Fig. 12 illustrates a method of decompressing an image which is 
encoded by the encoding method of Fig. 11 with an emphasis on a 
selected region of interest; 

Fig. 13 illustrates ROI coding when encoding and decoding are 
5 done on a block by block basis; 

Fig. 14 is a flow chart of another method of encoding data in 
accordance with the present invention; 

Fig. 15 illustrates an approach whereby a total number of bits to be 
used to encode the digital data representing the image is determined; 
10 Fig. 16 shows a block diagram of an embodiment of an encoding 

device of the present invention; 

Fig. 17 depicts a block diagram of an embodiment of a decoding 
device of the present invention; and 

Fig. 18 depicts an embodiment wherein the device includes a 
15 transmitting side that encodes the image and transmits the encoded data 
to a receiving side which receives and displays the image. 

nFTAll Fn DESCRI PTION OF THF PREFERRED FMRODIMENTS: 

The present invention is directed to a method and apparatus of 
20 performing image compression wherein a region of interest specified by the 
user is emphasized so that it is encoded with higher fidelity than the rest of 
the image. The emphasis can occur either during the beginning or from the 
middle of the encoding process. If it occurs during the middle of encoding, 
the emphasis can be driven by the use at the receiving side which receives 
25 portions of the encoded bit stream while the encoding is underway. The 
emphasis to the region of interest is done in the transform coefficient 
domain in order not to cause artificial boundaries around the region of 
interest on the reconstructed image. In an embodiment where the 
emphasis is done on the transform coefficients after the quantization, 
30 information ordering of the quantization indices corresponding to the region 
of interest is modified so that the region of interest can be reconstructed at 
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the earlier stages of progressive reconstruction; therefore, the region of 
interest is reconstructed with a higher fidelity at the low bit rate. Since the 
emphasis on the region of interest merely modifies the ordering with which 
the bit fields of the quantization indices are encoded, the emphasis does 
5 not cause any information loss. Also the modification of the information 
ordering is applicable not only by each coefficient level but also by each bit 
field level of the coefficient, which not only improves the quality of the 
specific parts of the image but also flexibly modifies the encoding ordering 
with which each part of the image is reconstructed. Another embodiment 
10 of this invention is to emphasize the transform coefficients before 
quantization. This embodiment does not provide such a flexible 
functionality as the other mode, but makes it possible to reconstruct the 
region of interest with higher fidelity than the rest of the image at any bit 
rate at a minimal increase of computational complexity. 
15 Fig. 1 illustrates a method for compressing an image with an 

emphasis on a selected region of interest in an image by allocating more 
bits to the region of interest or encoding the region of interest at the earlier 
stages of the encoding process than to the regions outside of the region of 
interest. Encoding method 100 comprises step 101 for performing a 
20 wavelet transform on the pixel values of the input digital image in order to 
represent the input image by a set of subbands consisting of the transform 
coefficients. Step 101 is followed by bit allocation step 102 and a 
quantization step 103. At step 102, a bit per coefficient (i.e., representation 
accuracy of the coefficient) is assigned to the transform coefficients in each 
25 subband in order to represent the transform coefficients with digitized 
values is determined in such a way that the subband which has higher 
variance or higher energy of the transform coefficients will be allocated a 
larger number of bits per coefficient, which is equivalent to being allocated 
a smaller quantization step size. However, in the case where the bit per 
30 coefficient for each subband or for all the subbands is predetermined, step 
102 is not performed. The allocated bits per coefficient are used in step 
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103. At step 103, quantization is performed on the transform coefficients in 
each subband in order to represent the transform coefficients of each 
subband with quantization indices whose representation accuracy is 
specified by the allocated bits per coefficient or by the quantization step 
5 size for each subband. Through step 103, quantization indices 
representing transform coefficients with a reduced or the same 
representation accuracy of the transform coefficient values is obtained. 
The obtained quantization indices are input to ROI coefficient scaling step 
107. 

10 Before, after, or together with steps 101, 102 and 103, region of 

interest selection step 104, ROI coefficient identification step 105, and ROI 
coordinate description step 106 are performed. At step 104, the region of 
interest is selected on the input image and the coordinates of the selected 
region of interest are input to steps 105 and 106. At step 105, wavelet 
15 transform coefficients corresponding to the selected region of interest, i.e., 
ROI coefficients, in each subband are identified in order to emphasize the 
selected region of interest in the image by emphasizing the ROI 
coefficients in each subband containing the wavelet transform coefficients. 
The identification result of the ROI coefficients (i.e., categories of the 
20 coefficients), which depicts whether the transform coefficients correspond 
to each region of interest or regions outside of the region of interest, is 
input to step 107. At ROI coordinate description step 106, coordinates of 
the selected region of interest are encoded in order to effectively transmit 
or store the ROI coordinate information, from which decoder can tell what 
25 region of interest is selected to be emphasized in the reconstructed image. 
The ROI description information is added to the header bits in the bit 
stream in transmission step 109. 

At ROI coefficient scaling step 107, among the quantization indices 
input from 103, only the quantization indices for the transform coefficients 
30 corresponding to the region of interest are emphasized in such a way that 
the quantization indices for the ROI coefficients are scaled up by a left bit 
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shift value (S) specified by a priority assigned to the region of interest so 
that the indices for the ROI coefficients are to be encoded as if the indices 
had larger values than their actual values. Therefore, they are encoded 
with a larger number of bits at a given bit rate or encoded at the earlier 
5 stages of the encoding process, at the following step 108 entropy 
encoding. The quantization indices, some of which" are scaled up. are 
input to step 108 together with the category of the coefficients, the 
identification result of ROI coefficients, formed at step 105 and the priority 
(the left bit shift value, S) used for the scaling up. 
10 At step 109 entropy coding, an entropy coding is performed on each 

bit element of the binary representation of the quantization indices in order 
to form an encoded data stream within which encoded bits generated from 
the bit fields at the higher bit significance level of the quantization indices 
are placed in the earlier portion of the bit stream than other encoding bits 
15 generated from the bit fields at the lower bit significance level. In other 
words, an entropy coding is performed on each bit field of the binary 
representation of the quantization indices in such an order that the bit field 
at the highest bit significance level (Most Significant Bit) is encoded first 
and bit fields at the decreasing order of bit significance levels are encoded 
20 in a decreasing order of bit significance levels. The entropy coding step 
can be terminated or suspended at any bit rate: when the bit budget for the 
encoded bit stream is used up, when the receiving side or storing side of 
the encoded bit stream does not need any further bits, when user or a 
system at the encoding side wants to terminate or suspend the step or 
25 when user or a system at the receiving side wants to terminate or suspend 
the step. 

The encoder avoids encoding bit fields of the bottom S least bit 
significance levels in the quantization indices for the ROI coefficients, 
because these bit fields, which do not exist before the scaling up of the 
30 ROI coefficients by S left bit shift, do not convey any information. 
Alternatively, in order to reduce the computational cost to avoid encoding 
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these bottom S bit fields, these fields whose values are uniformly filled with 
0 may be encoded together with the bottom S bit fields of the quantization 
indices for the regions out of the region of interest at the expense of 
increasing encoded bit rate. Top S bit fields of the quantization indices for 
5 the ROl coefficients are exclusively encoded without encoding any bit 
fields of the quantization indices for the regions outside of the region of 
interest in the same subband. Alternatively, in order to reduce 
computational cost to selectively encode the top S bit fields for the region 
of interest, these bit fields may be encoded together with the bit fields for 
10 the regions outside of the region of interest whose values are filled 
uniformly with 0 at the expense of increasing encoded bit rate. 

The preferable encoding technique in step 108 is either a bit plane 
coding such as a binary arithmetic coding technique or a zero tree coding 
such as the SPIHT coding technique. With a bit plane coding technique, all 
1 5 the bit fields at a certain bit significance level in each subband are encoded 
at the same encoding stage. After these bit fields are encoded, other bit 
fields in another bit significance level are encoded. In many cases, bit 
fields of the higher bit significance levels are encoded earlier than the bit 
fields of the lower bit significance levels in the same subband. In such 
20 cases, encoding results of the bit fields in the higher bit significance levels 
may be used for encoding the bit fields in the lower bit significance levels. 
With a zero tree coding technique, bit fields at the higher bit significance 
level in each quantization index are always encoded earlier than the bit 
fields at the lower bit significance level in each quantization index, but 
25 some of the bit fields at the lower bit significance level in the same 
quantization indices are encoded earlier than the bit fields at the higher bit 
significance level in other quantization indices. Encoded data formed at 
step 108 is sent to transmission step 109 where data bits and header bits 
are appended into a bit stream to be transmitted or stored. 
30 In a subband where the allocated bits per coefficient is smaller than 

the representation accuracy of the transform coefficients, each transform 
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coefficient is represented by a quantization index whose representation 
accuracy is smaller than the accuracy with which the value of the 
quantized transform coefficients are represented. In a subband where the 
allocated bits per coefficient is the same as the representation accuracy of 
5 the value of the transform coefficients, each transform coefficient may not 
be quantized and each coefficient value itself may be regarded as a 
quantization index to be given to ROI coefficient scaling step 107. This 
invention goes with any kind of quantization schemes where the larger 
transform coefficient is to be represented with the larger quantization 
1 o index. Preferred quantization in this invention is either a scalar quantization 
or a trellis coded quantization. With a scalar quantization, transform 
coefficients are quantized into indices based on the magnitudes of the 
coefficients with respect to a set of threshold values. With a trellis coded 
quantization, transform coefficients are quantized into indices not only 
1 5 based on the magnitude of themselves but also the states of the quantizer. 

In Figs. 3A-3F, ROI coefficient scaling step 107 is illustrated. ROI 
coefficient scaling is performed on the quantization indices of the transform 
coefficients either in each subband, in all the subbands or in several 
groups of subbands at a time. In a situation where the scaling is 
20 performed in each subband, each subband can be assigned a different 
priority including no priority (left bit shift value, S) to the quantization 
indices for the region of interest. In a situation where a selected region of 
interest needs to be emphasized in an image reconstructed only from 
some of the subbands, the ROI coefficient scaling needs to be performed 
25 only in the selected subbands (e.g., when a lower spatial resolution version 
of the image is reconstructed, ROI coefficients in the subbands which are 
not necessary for reconstructing the target spatial resolution are not scaled 
up). Hereafter, ROI coefficient scaling in each subband is disclosed, which 
can be generalized to be an ROI coefficient scaling to be performed in all 
30 the subbands or in several groups of subbands at a time, e.g., by assigning 
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the same priority value to the quantization indices for the region of interest 
in all the subbands or in several groups of subbands. 

To illustrate this concept, let us denote transform coefficients in one 
subband (subband[k]) as YQ) where j (0<= j < J) represents a coordinate of 
5 the transform coefficients and corresponding quantization indices in the 
subband[k], quantization index of the YQ) as Z(j), allocated bits per 
coefficient at the step 102 as N, bit fields of the binary representation of the 
quantization index Z(j) are b N _,(j), b N . 2 Q),..., and b 0 G), (b k (j), 0<=k<N, is 
either 0 or 1; b N . n G) is the bit field in the highest bit significance level of the 
10 Z(j)). Binary representation of the quantization index Z(j) is the following: 

Z(j) = 2 N - 1 x b N . n 0) + 2 N - 2 x b N . 2 (j) + + 2 1 x b,(j) + 2° x b 0 G). 

(Before the ROI coefficient scaling is performed, b n G) represents a bit value 
in the bit significance of 2 n .) 

When the transform coefficients identified as corresponding to the 
15 region of interest (i.e., ROI coefficients), Y(j) where j=js, js+1, ... and je, 

quantization indices Z(j) where j=js and je are the ROI coefficients in 

the subband[k], which are to be scaled up in step 107. When a priority 
assigned to the selected region of interest is a left bit shift value, S, 
quantization indices ZGs),..., and ZGe) are scaled up to be ZGs),.., and 
20 ZGe): 

ZGs) = 2 s x ZGs) = 2 N+S - 1 x b^Gs) + + 2 S+1 x b^s) + 2 s x b 0 Gs), 

ZsGs+1) = 2 s x ZGs+1) = 2 N+S1 x bM.,08+1) + + 2 S+1 x b^s+D + 2 s x 

boGs+1). 

25 ZsGe) = 2 s x ZGe) = 2 N+S1 x b^Qe) + + 2 S+1 x b,Qe) + 2 s x b 0 Ge). 

As a result of the scaling by left bit shift of S, a1) magnitude of the 
corresponding indices has become 2S times bigger. In other words, a2) bit 
significance level (sjevel) of each bit field has become bigger by S 
(s_level=N-1 -> s_level=N+S-1 , s_level=N-2 -> s_leveNN+S-2...., 
30 s_level=0 -> s_level=S). If each bit field is encoded in the order of 
decreasing order of bit significance level, each top S bit field of the scaled 
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up indices are to be encoded earlier than any other bit fields in the same 
subband. In other words, larger number of bit fields in the scaled up indices 
are to be encoded at the earlier stages of the encoding process. a3) In a 
situation where bit plane coding is used, encoding is done on each bit 

5 plane consisting of the bit fields of the same bit significance level. In each 
subband, each bit plane is encoded preferably in the decreasing order of 
bit significance level or any other order. Encoding ordering of each bit 
plane across subbands can be arbitrarily specified by following the 
encoding ordering within each subband. The following is an example of 

10 encoding ordering of each bit plane in the same subband: 
No 0. plane (b^Gs), b N . 1 (js+1),...,b N .iG e )}. 
No 1 . plane {b N . 2 Gs), b N . 2 (js+1 ),.... b N . 2 Ge)}, 

No S-1 . plane {b N _ s Gs), b N . s Gs+1) b N . s Ge)}, 

15 No S. plane {b N _ 1 (0),...,b N . 1 Gs-1), b N . s . 1 Gs),...,b N . s . 1 Ge). b N . l (je+1)..~.b N . l (J- 

1)}. 

No S+1. plane {b N . 2 (0),...,b N . 2 Gs-1), b N . s . 2 Gs) b N . s . 2 Ge), b N . 2 Ge+1) b N . 2 (J- 

D). 

20 No N-1. plane {b s (0),...,b s Gs-1), b 0 Gs),...,b 0 Ge), b s Ge+1),...,b s (J-1)}, 
No N. plane {b s . 1 (0),...,b s . 1 Gs-1). b^Ge+1) b s . 1 (J-1)}, 

No N+S-1 .plane {b 0 (0),...,b 0 Gs-1), b o 0e+1),..-.b o (J-1)}. 

The maximum left bit shift value, S max , is determined by the bit per 
25 coefficient assigned to the subband, by the maximum bit per coefficient 
among the ones assigned to all the subbands, by the maximum level of the 
significant bit in the subband, or by the maximum level of the significant bit 
in all the subbands. If a left bit shift value bigger than S max is specified, it 
can be adjusted to S max . In this case, the left bit shift value, S, is always in 
30 the following range: 0<=S<=S max (S ma = N, 0: no priority to the region of 
interest). Even if S max is not upper-bounded, the invention works at a small 
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increase of the encoded bit rate or a small increase of computational cost. 

If the maximum left bit shift or the value larger than the maximum 
left bit shift is chosen, all the bit fields of the quantization indices 
corresponding to the region of interest are to have a different bit 
5 significance level from all the bit fields of the quantization indices 
corresponding to the regions outside of the region of interest in the same 
subband. Thus, all the bit fields for the region of interest and all the bit 
fields for the regions outside of the region of interest are to be encoded 
separately. In other words, the quantization indices for the region of 
1 o interest and the quantization indices for the regions outside of the region of 
interest are to be separately encoded at the entropy coder. 

If the left bit shift value is less than the maximum value and larger 
than 0, the top S bit fields of the quantization indices corresponding to the 
region of interest are to be encoded separately from any bit fields of the 
15 quantization indices for the regions outside of the region of interest in the 
same subband, the rest of the N-S bit fields of the quantization indices 
corresponding to the region of interest are to be encoded at the same 
encoding stage with the top N-S bit fields of the rest of the indices in the 
same subband. and the rest of the S bit fields in the indices which 
20 correspond to the regions outside of the region of interest are to be 
encoded separately from any bit fields for the region of interest. In other 
words, the quantization indices for the region of interest and the 
quantization indices for the regions outside of the region of interest are 
partially separated to be encoded at the entropy coder, when the left bit 
25 shift is less than the maximum value and larger than 0. 

Preferable methods of ROI coefficient scaling are e1) scaling up the 
value of the quantization indices corresponding to the region of interest, 
e2) scaling up the bit significance levels of the bit fields associated with the 
region of interest e3) reassigning encoding ordering. e1), e2) and e3) are 
30 corresponding to the previously discussed a1), a2) and a3), respectively. 
Either the result of the ROI coefficient scaling up with e1), e2) or e3) at the 
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step 107 is used together with the priority to the selected region of interest 
and identification result of the coefficients corresponding to the region of 
interest at the step 108 in order to manage the entropy coding to be 
performed at the step 108 entropy coding. 
5 In Figs. 4A and 4B, ROI coefficients where multiple regions of 

interest are to be emphasized with different priorities are illustrated. When 
each selected region is emphasized with the same level of emphasis, 
quantization indices corresponding to every region of interest are to be 
scaled up with the same left bit shift value. In this case, the same ROI 
10 coefficient scaling is performed on the quantization indices corresponding 
to any regions of interest by the same way as illustrated in Figs. 3A-3F. 
When each selected region is emphasized with its own priority, scaling up 
illustrated in Figs. 3A-3F needs to be performed for each region of interest. 
In this case, transform coefficients corresponding to each region of interest 
15 selected at region selector step 104 have to be identified and categorized 
into a separate category in ROI coefficient identification step 105. In this 
categorization, transform coefficients corresponding to several regions of 
interest are looked upon the coefficients corresponding to a region of 
interest which is assigned the highest priority among all the selected 
20 regions of interest. Based on the category and priority assigned to each 
region of interest, ROI coefficient scaling is performed for each region of 
interest as is done in Figs 3A-3F. 

In Figs. 5A-5C, ROI coefficient scaling from the middle of encoding 
is illustrated. While the embodiment of ROI coefficient scaling method 
25 illustrated in Figs. 3A-3F, 4A and 4B either scales up the values of the 
quantization indices, scales up bit significance level of the every bit field of 
the quantization indices, or reassigns encoding ordering of every bit field of 
the quantization indices, the embodiment of the ROI scaling method 
disclosed in Figs. 5A-5C either scales up part of the value of the 
30 quantization index for a region of interest (the value consisting of some of 
the bit fields in the quantization index), scales up the bit significance level 
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of some of the bit fields in the quantization indices, or reassigns encoding 
ordering of some of the bit field of the quantization indices. In other words, 
while the ROI coefficient scaling method of Figs. 3A-3F, 4A and 4B uses all 
the bit fields in the quantization indices of the transform coefficients for a 
5 region of interest in order to emphasize the region of interest, the ROI 
coefficient scaling method of Figs. 5A-5C uses some of the bit fields in the 
quantization indices for a region of interest in order to emphasize the 
region of interest. Other than the aspect at which bit significance level or 
at which encoding stage the ROI coefficient scaling is used, the ROI 
10 coefficient scaling to be disclosed hereafter uses the same method as the 
ROI coefficient scaling in Figs. 3A-3F, 4A and 4B. 

In the ROI coefficient scaling method in Figs. 5A-5C, some of the bit 
fields of all the quantization indices for the transform coefficients are not 
scaled up. These bit fields can be some of the top bit fields, some of the 
1 5 bottom bit fields, or some of the middle bit fields. In this case, the encoded 
bit stream generated from the bit fields where ROI coefficient scaling is not 
used can be decoded with a decoding method which does not deal with 
region of interest. When the some of the top bit fields in each subband are 
encoded without using the ROI scaling, selection of the region of interest 
20 and selection of its priority can be specified by the receiving side if the 
encoded bit stream is transmitted to the receiving side during the encoding 
process and the encoder receives a feedback signal from the receiver: a 
user at a receiving side specify the region of interest and a priority on a 
partially reconstructed image from an incoming encoded bit stream from 
25 the encoder and feedback the coordinate information and priority of the 
region of interest to the encoder; then, encoder start ROI scaling in the 
middle of the encoding process. To explain the subject matter of Figs. 5A- 
5C, let us use the same terminology as used in disclosing the ROI 
coefficient scaling in Figs. 3A-3F. Transform coefficients in one subband 
30 (subband[k]): Y(j) where j (0<= j < J) represents a coordinate of the 
transform coefficients in the subbandfk]; quantization index of the Y(j): Z(j); 
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allocated bits per coefficient at step 102 are represented as N. The bit 
fields of the binary representation of the quantization index Z(j) are b N ..,G). 

b N . 2 G) and b 0 G). (b k (j) where 0<=k<N-1 is either 0 or 1; the left bit shift 

scaling value is represented by S; the number of bit significance levels 

5 where ROI coefficient scaling is not used is represented by P. In a situation 
where the top P bit planes are to be encoded without using ROI coefficient 
scaling, binary representation of the ZO) is described as the following: 

ZG) = 2 N - 1 x bN-1(j) + 2 N " 2 x b N . 2 G) + + 2 1 x b,(j) + 2° x b 0 (j). 

ZG) is represented by a combination of Z1Q) and Z2G): Z1G) is a portion of 

1 0 the ZG) which is as it is; Z2G) is a portion which is to be scaled up. 
ZG) = Z1G) + Z2G) or denoted as ZG) = { Z1G), Z2G) }, 
where 

Z1Q = 2 N " 1 x b N .-,G) + 2 N 2 x b N . 2 G) + + 2 N P x b N . P (j). 

Z2G) = 2 N P " 1 x b N . P . n G) + 2 N P " 2 x b N . P . 2 G) + + 2° x b 0 G). 

15 Suppose the transform coefficients identified as corresponding to 

the region of interest in step 105 are YG) where j=js, js+1 and je, and 

the corresponding quantization indices to be scaled up in the step 107 are 
ZG) where j=js, .... and je, 

Z2(), scalable portion of the quantization indices of ZGs),..., ZGe), are 
20 scaled up to be Z2sGs),..., Z2sGe): 

Z2sGs) =2 S x Z2Gs) = 2 N+S " P - 1 x b N . P ,Gs) + 2 N+SP " 2 x b N . P . 2 Gs) + + 2 s x 

b 0 Gs), 

Z2sGs+1)=2 s x Z2Gs + 1) =2 N+S - P1 x b^Qs+D + 2 N+S " P " 2 x b N . P . 2 Gs+1) + + 

2 s xb 0 Gs+1), 

25 

Z2sGe) = 2 s x Z2Ge) = 2^ x b N . P ,Ge) + 2 N+SP 2 x b N . P ,Ge) + + 2 s x 

b 0 Ge). 

With the ROI coefficient scaling from the middle of encoding, 

ZGs),-, and ZGe) are converted from ZGs)={Z1Gs), Z2Gs)} and 

30 ZGe)={Z1Gs), Z2Ge)} to ZsGs)={Z1Gs),Z2sGs)} and ZsGe)={Z1Ge), 

Z2sGe)}. In the ROI coefficient scaling method shown in Figs. 5A-5C, Z2G) 
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instead of ZO) is scaled up with the same scaling method as in Figs. 3A-3F, 
4A and 4B. 

The maximum left bit shift scaling value, S max , is determined by the 
bit per coefficient assigned to the subband, N, and by the number of bit 
5 fields in each quantization index which are to be encoded without using 
ROI coefficient scaling, P. In this case, the left bit shift value, S, is in the 
following range: 0<=S<=S max (S ma = N-P. 0: no priority to the region of 
interest). 

If the maximum left bit shift, S max = N-P, is chosen, bottom N-P bit 
1 0 fields of the quantization indices corresponding to the region of interest are 
separated from the bottom N-P bit fields of the quantization indices which 
correspond to the regions outside of the region of interest in the same 
subband. Thus, the bottom N-P bit fields for the region of interest and for 
the regions outside of the region of interest are to be separately encoded. 
15 |f the left bit shift value is less than the maximum value and larger 

than 0, S bit fields of the quantization indices for the region of interest at 
the <P+1) st highest significance level through the (P+S) ,h highest bit 
significance levels are separate from any bit fields for the regions outside 
of the region of interest (the 1 st highest significance level: MSB, the N ,h 
20 highest significance level: LSB in this example). The rest of the N-P-S bit 
fields of the^quantization indices corresponding to the region of interest are 
to be encoded together with the N-P-S bit fields for the regions outside of 
the region of interest at each bit significance level. The rest of the S bit 
fields, S bottom bit fields in the quantization indices for the regions outside 
25 of the region of interest are separated from any bit fields for the region of 
interest. 

In Figs. 6A-6C illustrate a ROI coefficient scaling method where 
multiple regions of interest are emphasized from different stages of 
encoding with different priority. This technology is equivalent to repeatedly 
30 applying the ROI coefficient scaling method from the middle of the 
encoding discussed in Figs. 5A-5C in order to emphasize another region of 
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interest in the middle of the encoding process where some of the 
quantization indices are already scaled up from the beginning or at some 
stage of the encoding process by the methods illustrated in Figs. 3A-3F, 
4A-4F, or 5A-5C. This method makes it possible to add different regions of 
5 interest or expand the region of the already selected region of interest in 
the middle of the encoding process; or reconstruct the selected region of 
interest part by part from different stages of the encoding process by 
dividing the region into several parts and assigning different priorities to 
each part. Also in this case, selection of another region of interest and 
10 selection of its priority can be done by the receiving side if the encoding 
process and decoding process at the receiving side are done interactively. 

In Figs. 7A-7D, entropy coding step 108 of the quantization indices 
through ROI coefficient scaling step 107 is illustrated when a bit plane 
based coder is used. A characteristic of the present invention is to modify 
15 a set of the quantization indices which is an input to a bit plane coder at 
each bit significance level or modifying an encoding ordering of the bit 
fields in each quantization index by using the result of the ROI coefficient 
scaling in step 107. The result of the ROI coefficient scaling defines each 
bit plane to be encoded with a bit plane coder and encoding ordering of 
20 each bit plane. Determining encoding ordering based on the bit 
significance level of each bit plane is the simplest approach. Determining 
separate encoding ordering for each bit plane corresponding to each 
category of coefficients is another approach. For example, encoding 
ordering for the bit planes consisting of the coefficients corresponding to 
25 the region of interest can be determined separately from the encoding 
ordering for the bit planes consisting of the coefficients corresponding to 
the regions outside the region of interest. If separate encoding ordering is 
used for each category of coefficients, encoding ordering across each 
category can be arbitrarily selected and encoding ordering for the entire 
30 image can be more flexibly specified. When bit plane coder which 
encodes each bit plane without using any encoding result or information of 
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the other bit planes, encoding ordering of each bit plane can be arbitrary. 
The bit plane coder used in this embodiment can be any binary entropy 
coder such as a binary arithmetic coder. Bit plane coding can be 
performed on each bit plane defined in each subband or on each bit plane 
5 defined in each group of subbands, possibly on each bit plane defined in 
all the subbands. 

Figs. 7A-7D illustrate a case where bit plane coding is performed in 
each subband where ROI coefficient scaling is used on every bit field as 
shown in Figs. 3A-3F, 4A and 4B. ROI coding starts from the beginning of 
10 the encoding. In the entropy coding in Figs. 7A-7D, each bit field of the 
quantization indices for the transform coefficients which has the same bit 
significance level determined by the ROI coefficient scaling in step 107 is 
grouped together and forms a bit plane. Alternatively, each bit field which 
is assigned the same encoding or information ordering in step 107 is 
15 grouped into the same bit plane. When the bit per coefficient (i.e. 
representation accuracy of the indices) is N and the priority to the selected 
region or regions of interest is S (0<=S<=N), the number of the bit planes 
becomes N+S. S bit planes consisting of the bit fields at the S highest bit 
significance levels are related to the coefficients for the region of interest. 
20 The next N-S bit planes consisting of the bit fields in the next N-S bit 
significance levels are related to the coefficients for the entire image. The 
last S bit planes consisting of the bit elements in the lowest S bit 
significance levels are related to the coefficients for the regions outside of 
the region of interest. Each bit plane is encoded by the bit plane coder bit 
25 plane by bit plane. Bit planes in the higher significance level are encoded 
at the earlier stages of the encoding process than the bit planes in the 
lower significance levels. An encoded bit stream of each bit plane is 
appended to an encoding tag which is used to identify which bit 
significance level the generated portion of the encoded bit stream 
30 represents. Together with the priority, i.e., left bit shift value to the selected 
region of interest, the tag shows whether the bit portion represents only the 
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region of interest, the entire image, or the regions outside of the region of 
interest. When the priority to the region of interest is the maximum, N, 
each bit portion generated from each bit plane consists either of the region 
of interest or the regions outside of the region of interest. When the priority 
5 is the maximum, bit planes for the region of interest and those for the rest 
of the region are independently encoded. Also, encoding ordering and the 
ordering of the bit portions of the encoded bit stream between the region of 
interest and the rest of the regions can be arbitrary. 

By counting the number of the bits spent in the encoding of each bit 
10 plane and by terminating the encoding process when the bit budgets for 
the region of interest, the regions outside of the region of interest, or both 
is exceeded, the number of the bits allocated to the region of interest and 
to the regions outside of the region of interest is more precisely controlled. 
If the priority to the region of interest is the maximum, the number of 
1 5 allocated bits for the region of interest and for the rest of the region can be 
controlled independently. 

When encoding a bit plane consisting only of the bit fields 
corresponding to the region of interest or a bit plane consisting only of the 
bit fields corresponding to the regions outside of the region of interest, bit 
20 fields in each bit plane are either ordered into a one dimensional signal to 
be entropy encoded by using context modeling for a one dimensional 
signal or kept as they are as a two dimensional signal to be encoded by 
using context modeling for a two dimensional signal. When each bit field is 
to be encoded as two dimensional signal, bit fields on the coordinates out 
25 of the region of interest or bit fields on the coordinates within the region of 
interest do no exist in some bit planes in some bit significance levels. In 
order not to increase the encoded bit rate, these coordinates are skipped 
to be encoded in the encoding process. Alternatively, in order to reduce 
computational complexity of skipping such coordinates, these coordinates 
30 are not skipped encoding but encoded as if they had bit fields whose 
values are 0. These 0 values are discarded during the decoding process. 
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Instead of 0, 1 can be used as the assumed value. 

Priority to the region of interest can be modified during the entropy 
coding step. One example is to reduce the priority to the region of interest 
during the entropy encoding step: if number of bits or bit rate spent for the 
5 region of interest reaches a pre-determined value, priority value is reduced 
in order not to emphasize the region of interest for the rest of the encoding 
process. Alternatively, if the estimated MSE or peak signal-to noise ratio 
(PSNR) value of the region of interest in the wavelet domain reaches a 
predetermined value, priority value can be reduced. If the priority is 
10 controlled according to feedback from the decoding side, not only the 
estimated MSE or PSNR, but also the MSE or PSNR calculated on the 
partially reconstructed image can be used. Priority control of the region of 
interest, based upon the bit-rate, MSE, or PSNR, or any other 
measurement associated with compression performance (e.g. bit-rate 
1 5 versus MSE) of the region of interest, makes it possible for the encoding or 
decoding system to determine the most appropriate ROI coding strategy. 
Furthermore, the user at the encoding or the decoding side may decide on 
the strategy. The ROI coding strategy determines when and how much the 
ROI coefficient scaling is performed, how much of the bit rate is allocated 
20 for the region of interest and for the rest of the image to be encoded with 
entropy encoder, when the encoding process of the region of interest is 
terminated, and any other information used for encoding or decoding the 

region of interest. 

In Figs. 8A-8C, the case where bit plane coding is performed in 

25 each subband where ROI coefficient scaling is used only on the bit fields at 
some bottom bit significance levels as shown in Figs. 5A-5C and 6A-6C 
(ROI coding starts from the middle of the encoding) is disclosed. In a 
situation where the bits per coefficient assigned to a subband (i.e. 
representation accuracy of the indices) is N, ROI coefficient scaling is used 

30 after the top P bit planes of the quantization indices, and the priority to the 
selected region or regions of interest is S (0<S<=N-P), the number of the 
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bit planes in the subband is P + N-P+S = N+S. The top P bit planes 
consist of the top P bit elements in each quantization index. Therefore, the 
top P bit planes represent the entire image. From the bottom N-P bit fields 
in each quantization index, N-P+S bit planes are formed by the same 
5 method as in Figs. 6A-6C (grouping the bit elements in the same bit 
significance level after scaled up in step 107): top S bit planes among N- 
P+S bit planes represent the selected region or regions of interest, next N- 
P-S bit planes represent the entire image, and the last S bit planes 
represent the regions outside of the region of interest. P bit planes where 
10 ROI coefficient scaling is not used do not have to be the top P bit planes in 
each subband. They can be bottom P bit planes or middle P bit planes. 
When they are bottom P bit planes, ROI coefficient scaling as in Figs 3A- 
7F and entropy coding as in Figs. 7A-7D are performed on the top N - P bit 
planes in each quantization index. 
15 as in Figs. 7A-7D, an encoded bit stream for each bit plane or an 

encoded bit portion of each bit plane is appended to a tag and 
concatenated to form an encoded bit stream of the subband. Also, by 
counting the number of bits spent in each bit plane, or by counting the 
number of bits spent in the top P bit planes representing the entire image, 
20 the next top S bit planes representing the region of interest, the next N-P-S 
bit planes representing the entire image and the rest of the S bit planes 
representing the regions out of the region of interest, the number of the bits 
which will be spent for the region of interest and the rest of the regions can 
be more precisely controlled. 
25 The bit significance level at which the ROI coefficient occurs can be 

specified during the entropy encoding process by the bit-rate, estimated 
MSE or PSNR in the wavelet domain as well as the priority to the region of 
interest. If the bit-rate spent for encoding some of the bit planes without 
using the ROI coefficient scaling technique reaches a certain value, the 
30 scaling technique starts to be used with higher priority from the bit plane to 
be encoded immediately after the bit plane currently under encoding. If the 
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bit rate spent for encoding the bit planes representing the region of interest 
reaches another certain value, priority to the region of interest is reduced in 
order not to emphasize the region of interest for the rest of the encoding 
process. Instead of the bit rate, the MSE value estimated in the wavelet 
5 domain for the entire image or for the region of interest can be used to 
determine from which bit plane ROI coefficient scaling is used and from 
which bit plane priority to the region of interest is reduced. 

At bit allocation step 102, bits per coefficient assigned to each 
subband are determined in order to reduce the distortion of the entire 
10 reconstructed image as much as possible at a given bit rate for the entire 
image. Bits per coefficient or bit rate for each coefficient in each subband 
is determined in such a way that the subband which has higher variance or 
higher energy of the transform coefficients will be allocated the larger 
number of bits per coefficient. The bit allocation result given to 
15 quantization step 103 can either be a number of bits per coefficient in each 
subband, quantization step size, or a parameter to choose a quantization 
scheme at step 103. When bits per coefficient, a quantization step size or 
a quantization scheme is predetermined in each subband, for example, 
when lossless coding is realized, bit allocation is not performed. 
20 At region selector step 104, a region or regions of interest are 

selected by the user on the image displayed on a display. The displayed 
image is either a full spatial resolution image to be encoded or one of the 
lower resolution versions of the image. If the region selection occurs on a 
lower resolution version of the image, coordinates of a corresponding 
25 region or regions of interest in the full resolution image to be encoded are 
calculated to be used for identifying the transform coefficients 
corresponding to the region of interest in the full resolution image. The 
region selection can also be performed by a method wherein an automatic 
target recognition system defines a region or regions of interest based on a 
30 set of criteria. The automatic target recognition system may utilize a 
variety of approaches to identify the region of interest. For example, 
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pattern recognition software may be used to identify objects of interest in 
the image. 

At ROI coordinate description step 106, coordinates to describe 
each region of interest are encoded to be efficiently transmitted or stored 
5 as overhead information of the encoded bit stream of the image. When the 
selected region of interest is of rectangular shape/two coordinates at both 
edges of a diagonal line of the rectangle will describe the region. When 
the selected region of interest is of a circular shape, the center coordinate 
of the circle and the length of the radius will describe the region. When the 
10 selected region or regions are a union of several rectangles or circles, 
each of the rectangles or circles are described in the method discussed 
above. If the selected region of interest is of arbitrary shape, the 
boundaries of the region are encoded by any shape encoding method such 
as a chain coding method or an object coding method. 
15 At transmitter step 109, portions of the encoded bit stream 

generated at entropy coding step 108 can be ordered into any order. If the 
encoded data is to be transmitted to a display environment at which a 
lower resolution version of the image is most frequently reconstructed, 
portions of the encoded bit stream associated with the spatial resolution 
20 are ordered in the earlier portions of the whole encoded bit stream than the 
portions of the encoded bit stream associated with the higher spatial 
resolutions. This ordering of the portions of the bit stream re-orders the bit 
portions if the encoding ordering with which the bit portions are generated 
at the entropy coding step and transmission ordering of the bit portions are 
25 different. 

At ROI coefficient identification step 105, transform coefficients 
corresponding to the region of interest selected in the image domain are 
identified either through tracing the inverse wavelet transform from the 
image domain to the transform domain or tracing the inverse wavelet 
30 transform means to identify from which transform coefficients each pixel 
value is reconstructed through filtering and up sampling performed on the 
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transform coefficients. In this identification process, filter coefficients do 
not play any role. Instead, filter length or filter support area plays a role. 
The larger the filter support, the larger the number of transform coefficients 
that will correspond to a pixel in the image. This identification process is 
5 done through each level of an inverse wavelet transform. Also, this 
identification is performed along each dimension (vertical direction or 
horizontal direction) of the image or the subbands. Alternatively, the 
identification can be performed by tracing the forward wavelet transform 
from the image domain to the transform domain through each level of 
10 wavelet transform comprising filtering and down sampling. 

Let us denote the image as X(k 0 ) where 0<= k 0 <K, a 1st level 
decomposed lowpass subband as L n (kl_ t ) and a highpass subband H,(kH,) 
where 0 <= kL 1t kH, < K/2, a 2nd level decomposed lowpass subband as 

L 2 (kL 2 ) and a highpass subband as H 2 (kH 2 ) where 0<= kl_ 2 , kH 2 <K/2 2 , 

15 Nth level decomposed lowpass subband as L N (kL N ) and a highpass 
subband as H N (kH N ) where 0<= kl_ N . kH N <K/2 N . Therefore, a wavelet 
transform, mallet type wavelet decomposition or dyadic wavelet 
decomposition is: X is reconstructed from L, and H, through 1 level inverse 
wavelet transform; L t is reconstructed from L 2 and H 2 ; Thus, X is 
20 reconstructed from H 1t H 2 , .... H N , L N ; L„ L 2 , ... and U, are subbands by 
way of which each level of wavelet transforms and inverse wavelet 
transforms are performed. 

When k^RCko.Rs <= ko_R <= k 0 _Re) is a pixel in the selected 
region of interest (ko_R: pixel coordinate in the region of interest; k 0 _Rs 
25 and k 0 _Re: coordinates for the pixels on the boundaries of the region of 
interest; k 0 _R can be a single pixel in the region of interest (i.e., k 0 _R = 
k 0 _Rs = ko_Re); in the following explanation, the image and region of 
interest are assumed to be a one dimensional signal, thus, k 0 _Rs is the 
smallest coordinate and kO_Re is the largest coordinate in the region of 
30 interest), transform coefficients corresponding to X(k 0 _R) are identified in 
the L,() and H,() such that 
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- ROI coefficients in L.,0: kL,JRs <= kl_ n <= kL n _Re, 

- ROI coefficients in H^): kH,_Rs <= kH-, <= kH,_Re. 

Then, by assuming that the 1st level decomposed low pass signal, 
L^kL,). is an image and kl_ n _Rs <= kL, <= kl_ t _Re are the region of 
5 interest, transform coefficients corresponding to L^kL,) where kL^Rs <= 
kL, <= kL,_Re are identified in the L 2 () and H 2 (). 

- ROI coefficients in L 2 (): kL^Rs <= kl_ 2 <= kL^Re, 

- ROI coefficients in H 2 (): kH^Rs <= kH 2 <= kH^Re. 

This process is repeated until the subbands L N and H N to complete 
10 the ROI coefficient identification for the pixel k 0 _R in the image X(). 

As shown in the above, ROI coefficient identification can be 
performed separately for each pixel in the selected region of interest. 
Thus, the ROI coefficient identification is independent of the shape of the 
region of interest. However, this identification process does not have to be 
15 performed for every pixel in the region of interest. Instead, this 
identification process is performed only on the pixels on the boundaries of 
the region of interest and on the pixels which are located one pixel inside 
the boundaries of the region of interest as shown in the Fig. 9. Since the 
number of the transform coefficients corresponding to each pixel may 
20 change based on the location of the pixel because of up sampling 
(inserting 0 between each coefficient before performing filtering in every 
1st level inverse wavelet transform) or down sampling (every other 
coefficient is discarded after filtering is performed in every 1st level wavelet 
transform), transform coefficients corresponding to the pixels on the 
25 boundaries may not be on the boundaries of the corresponding region 
consisting of the ROI coefficients in each subband. Instead, transform 
coefficients corresponding to the pixels which are located one pixel inside 
the boundaries of the region of interest may be on the boundaries of the 
corresponding region in each subband. Pixels on the boundaries and the 
30 pixels one pixel inside the boundaries are necessary and sufficient to 
identify the transform coefficients which form boundaries of the 
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corresponding region in each subband. Alternatively, if both low-pass filter 
and high-pass filter have even numbered filter length, only the pixels on the 
boundaries are used to identify the coefficients which form the boundaries 
of the corresponding region in each subband. 
5 Based on the identified results for the boundaries and one pixel 

inside the boundaries, the rest of the transform coefficients are identified 
by choosing every coefficient within the boundaries of the corresponding 
region in each subband. 

Suppose X(k 0 ) (ko_Rs =< k 0 =< k 0 _Re) are pixels in the selected 
10 region of interest (k 0 _Rs and k 0 _Re: coordinates for pixels on the 
boundaries of the region of interest; , i.e., the smallest coordinate in the 
region of interest, k 0 _Re: coordinate for a pixel on the other boundaries of 
the region of interest; in the following explanation, the image and region of 
interest are assumed to be a one dimensional signal, thus, k 0 _Rs is the 
15 smallest coordinate and k 0 _Re is the largest coordinate in the region of 
interest). 

Left boundaries of the corresponding region are identified as the 
following: transform coefficients corresponding to X(k 0 _Rs) are identified in 
the L,() and H^) such that 
20 - ROI coefficients in L,(): ksL,_Rs <= kl_ n <= ksL^Re, 

- ROI coefficients in H,(): ksH,_Rs <= kH, <= ksH n _Re. 

Also, transform coefficients corresponding to X(k 0 _Rs-1) are identified such 

that 

- ROI coefficients in L,(): ksL^R's <= kL, <= ksL^R'e, 
25 - ROI coefficients in H,Q: ksH^R's <= kH, <= ksH^R'e. 

The smaller one of the ksL n _Rs and ksL,_R's is on the left boundaries of 
the corresponding region in L(), and the smaller one of the ksH^Rs and 
ksH,_R's is on the left boundaries of the corresponding region in H(). 

Right boundaries of the corresponding region are identified as the 
30 following: transform coefficients corresponding to X(k 0 _Re) are identified in 
the L,() and H^) such that 
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- ROl coefficients in !_,(): keL^Rs <= kl_ n <= keL^Re, 

- ROl coefficients in H^): keH^Rs <= kHi <= keH^Re. 

Also, transform coefficients corresponding to X(ko_Re-1) are identified such 

that 

5 - ROl coefficients in !_,(): keL^R's <= kL, <= keL,_R'e, 

- ROl coefficients in H',0: keH n _R's <= kH, <= keH^R'e. 

The larger one of the kel_ n _Re and keL^R'e is on the right boundaries of 
the corresponding region in L(), and the larger one of the keH,_Re and 
keH^R'e is on the right boundaries of the corresponding region in H(). 
10 Alternatively, ROl coefficient identification is performed on each 

pixel in the region of interest. A set of transform coefficients corresponding 
to each pixel in the region of interest is identified by tracing either a set of 
wavelet transforms which are performed on the pixel or a set of inverse 
wavelet transforms which reconstruct the pixel value. The identified set of 
15 coefficients in each subband corresponding to each pixel in the region of 
interest are categorized into a sub-category which belongs to the category 
of coefficients corresponding to the region of interest so that the 
identification result can be used to scale up and down or reconstruct the 
whole region of interest as well as to scale up and down or reconstruct an 
20 arbitrary part of the region of interest. Some identified coefficients belong 
to plural sub-categories because of the overlap of the low pass filter or high 
pass filter used in the wavelet or inverse wavelet transform. Based on the 
number of pixels to which each identified coefficient corresponds, each 
sub-category can be divided into sub-sub-categories. Ultimately, each 
25 identified coefficient can have an attribute depicting which pixels and how 
many pixels within the region of interest the coefficient corresponds to. 
The attribute can also depict how many pixels outside of the region of 
interest the coefficient corresponds to in addition to pixels within the region 
of interest. 

30 As shown in Figs. 10A-10B, a wavelet transform is accomplished by 

a low pass filter whose filter coefficients are g A (k) and a high pass filter 
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whose filter coefficients are f A (k) and down a sampler which discards every 
other pixel or transform coefficient. A one level wavelet decomposition for 
one dimensional signal is to decompose a signal X into a low pass 
subband L and a high pass subband H. The low pass subband L is 
5 obtained by applying low pass filtering to the signal X and down sampling 
the low pass filtered signal. The high pass subband H is obtained by 
applying high pass filtering to the signal X and down sampling the high 
pass filtered signal. 

If the signal is a two dimensional signal such as an image, one level 
10 decomposition is to perform the one level decomposition for a one 
dimensional signal on X in either horizontal or vertical direction in order to 
obtain L and H. Then the same decomposition for a one dimensional 
signal is performed on the L and H in another direction in order to obtain 
LL1 and LH1, and HL1 and HH1, respectively. If the same decomposition 
15 is performed on LL1, LL2, LH2, HL2, HH2 are obtained and X is 
decomposed into LL2, LH2, HL2, HH2, LH1, HL1 and HH1. 

A subband decomposition where all the decomposition is done by 
performing a one level decomposition only on the LL subband is called a 
mallet wavelet decomposition, or just a wavelet decomposition. Subband 
20 decomposition where a one level decomposition is repeated equally on 
each subband is called spacl wavelet decomposition. Subband 
decomposition where a one level decomposition is repeated arbitrarily on 
each subband is called wavelet packet decomposition. 

An inverse wavelet transform is accomplished by an up sampler 
25 which inserts 0 between each pixel or coefficient and a low pass filter 
whose filter coefficients are g s (k) and a high pass filter whose filter 
coefficients are f s (k). A one level wavelet composition for a one 
dimensional signal is to compose L and H into X. In this one level 
composition, L is up-sampled by 2 and low pass filtered, H is up-sampled 
30 by 2 and low pass filtered and then both of the up-sampled and filtered L 
and H are added to compose X. This one level composition is performed 
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on every decomposed subband to perform any level and any type of 
inverse wavelet transform. 

In Fig. 2, a method of decompressing an image which is encoded 
with an emphasis on the selected region of interest in Fig. 1 is illustrated. 

5 In Fig. 2, the decoding method 200 comprises a receiver step 201 for 
receiving an encoded bit stream consisting of header bits and "data bits. 
Step 201 is followed by region coordinate decoding step 202 and entropy 
decoding step 204. At step 202, encoded data of the region coordinates 
are decoded in order to obtain the coordinates of the region of interest 
10 (ROI) which is to be reconstructed with an emphasis in the reconstructed 
image. The decoded ROI coordinates are given to ROI coefficient 
identification step 203 where wavelet transform coefficients corresponding 
to the region of interest are identified. The ROI coefficient identification 
result is used at entropy decoding step 204 and ROI coefficients de-scaling 

15 step 205. 

At entropy decoding step 204, entropy decoding is performed on the 
incoming bit stream with the data bits which are received at receiver step 
201. The decoded bits are obtained for each bit field of the binary 
representation of the quantization indices of each transform coefficient. 
20 Within each quantization index, the bit field of the higher bit significance 
level is decoded earlier than the bit field of the lower bit significance level. 
In other words, entropy decoding is performed to obtain each bit field within 
a quantization index in a decreasing order of bit significance. In order to 
know which quantization indices are scaled up as the coefficients 
25 corresponding to the region of interest, and also how much they are scaled 
up, the entropy decoding process refers the ROI coefficient identification 
result at step 203 and takes out the priority value assigned to the region of 
interest from the header bits received at receiver step 201 . The entropy 
decoder used in step 204 is either bit plane decoding technique or SPIHT 
30 decoding technique: if entropy encoding is done with a bit plane coding 
technique, decoding has to be done with a corresponding bit plane 
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decoding technique; if entropy encoding is done with a SPIHT coding 
technique, decoding has to be done with a SPIHT decoding technique. 
Entropy decoded bits are organized as the bit field values of the binary 
representation of the quantization indices and are given to ROI coefficient 
5 de-scaling step 205 as the quantization indices. 

At step 205, the quantization indices corresponding to the region of 
interest are scaled down in order to perform de-quantization on the 
quantization indices at step 206. Since the quantization indices scaled up 
in the encoding process are to be scaled down by the same amount of bit 
10 shift value as the value used in the encoding process, no information is lost 
due to the scaling down. Which quantization indices are to be scaled down 
is given by the ROI coefficient identification result at step 203, and how 
many bit shifts the indices are scaled down is taken out from the header 
bits received at step 201 . 
-15 At de-quantization step 206, de-quantization is performed on the 

quantization indices in order to obtain de-quantized transform coefficients 
in each subband. The de-quantization scheme is specified by the 
quantization step size, bits per coefficient, or quantization table, either of 
which is taken out from the header bits. 
20 At inverse wavelet transform step 207, an inverse wavelet transform 

is performed on the de-quantized transform coefficients in each subband in 
order to obtain a reconstructed image. The number of wavelet 
decomposition levels and the type of wavelet decomposition are provided 
by the header bits taken out from the encoded bit stream at step 201 . The 
25 number and the type have to be the same ones as used in the encoder. 

At region coordinate decoding step 202, encoded data regarding the 
coordinates of the region of interest formed at ROI coordinate description 
step 106 in encoding method 100 is decoded and ROI coordinates are 
obtained. The step is performed by performing the step 106 ROI 
30 coordinate description in a reverse order. In other words, step 202 is to 
perform step 106 in such a way that the output of step 106 is to be the 
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input to step 202 and the input to step 106 is to be the output of step 202. 
In case multiple regions of interest are emphasized in the encoded bit 
stream, step 202 decodes coordinates for each region of interest. 

At ROI coefficient identification step 203, the same process as step 
105 in the encoding method 100 is performed. The reason why the same 
process is repeated at the decoding side is that the information specifying 
the selected region of interest within the encoded bit stream can be 
transmitted or saved more efficiently in the form of a description in the 
image domain, which is provided as the input to step 202, than in the form 
of a description in the wavelet domain which is obtained through step 203. 
In case multiple regions of interest exist in the encoded bit stream, ROI 
coefficients corresponding to each region of interest are identified and 
categorized in each category. 

At entropy decoding step 204, the encoded bit stream generated at 
the step 108 entropy encoding in the encoding method 100 is decoded. 
When a bit plane encoder is used at step 108, a corresponding bit plane 
decoder is used in step 204. Step 204 is to perform entropy decoding in 
the reverse order of step 108 entropy coding in encoding method 100. 
This decoder can deal with multiple regions of interest starting from 
different stages of encoding with different priority, ROI coding starting from 
the beginning of the encoding, and ROI coding starting from the middle of 
encoding. 

At step 204, the encoded data bit stream is separated into a set of 
bit streams for each subband by seeking an encoding tag for each 
subband. Then, each bit stream for each subband is separated into a set 
of bit streams for each bit plane by seeking an encoding tag for each bit 
plane. Each bit stream for each bit plane is entropy decoded by a bit plane 
decoder. The bit plane decoding is done in the order of decreasing bit 
significance level. The number of bit planes in each subband is given by a 
summation of the bits per coefficient for each subband and the priority (left 
bit shift value) assigned to the region of interest. To which bit field each 
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decoded bit is assigned is determined by the ROI coefficient identification 
result obtained at step 203 together with the number of bit planes and 
priority to the region of interest. 

The bit rate for each subband can be controlled at step 204. The 
5 simplest way is to truncate the bit stream for each subband at a desired bit 
rate or at a desired number of bits. Thanks to the priority control to the 
region of interest and specifying the starting stage of emphasizing the 
region of interest at the encoding side, bit allocation for the region of 
interest and for the rest of the image at a given bit rate can be roughly 
10 controlled together with the truncation of the stream for each subband. 
However, if the bit rate for the stream of a set of bit planes representing the 
region of interest is controlled with a one rate controller (1), the bit rate for 
the stream of a set of bit planes representing both the region of interest 
and the rest of the image is controlled with a one rate controller (2), and 
15 the bit rate for the stream of a set of bit planes representing the regions out 
of the region of interest is controlled with another rate controller (3), the bit 
rate allocated to the region of interest and the rest of the image is more 
precisely controlled at a given bit rate for the entire image. If the priority to 
the region of interest is the maximum, all the bit planes represent either the 
20 region of interest or the rest of the image. Thus, providing the rate 
controller for (1) and (3) makes it possible to control the bit rate for the 
region of interest and also the rate for the rest of the image by bit accuracy. 

At the ROI coefficient de-scaling step 205, the values of the 
quantization indices corresponding to the region of interest or the bit 
25 significance levels of the quantization indices corresponding to the region 
of interest are scaled down. 

The details of this step are exactly the same as the details of ROI 
coefficient scaling step 107 in the encoding method 100 except that the 
input to step 107 is the output of step 205, output of step 107 is the input to 
30 step 205, scaled up in step 107 corresponds to scaled down in step 205. 
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Also, the encoding ordering in step 107 corresponds to the decoding 
ordering in step 205. 

At de-quantization step 206, the de-quantization scheme which 
restores the transform coefficients quantized by the quantization scheme 
5 utilized in step 103 in encoding method 100 is used. If the quantization is 
done by a scalar quantization at the encoder, the de-quantization has to be 
done with a scalar de-quantization. If the quantization is done with a trellis 
coded quantization, the de-quantization has to be done with a trellis coded 
de-quantization. Representation levels of the de-quantizer can be mid- 
10 points of the decision levels in the quantizer or can be centroids of the 
decision levels which are calculated by assuming the distribution of the 
quantization index values for each subband. Even if quantization is not 
performed at the encoder, de-quantization is performed at the decoder 
except when an integer wavelet transform is used and both encoding and 
15 decoding is done losslessly. When encoding is done losslessly and 
decoding is done with some loss, quantization is not done in the encoding 
process but de-quantization is done in the decoding process. 

At inverse wavelet transform step 207, an inverse wavelet transform 
which reconstructs the image decomposed by the wavelet transform 
20 utilized in step 101 in the encoding method 100 is used. 

In Fig. 11, a method for compressing an image with an emphasis on 
a selected region of interest in an image by allocating more bits to the 
region of interest than to the regions out of the region of interest is 
illustrated. The major difference between this encoding method and the 
25 method in Fig. 1 is that the ROI coefficient scaling is performed before 
quantization is performed on the transform coefficients in this method. The 
steps different from those in Fig. 1 are the step for ROI coefficient scaling 
and the step for entropy coding. The rest of the steps are the same. 

In Fig. 11, encoding method 1100 comprises wavelet transform step 
30 1101 for performing a wavelet transform on the pixel values of the input 
digital image in order to represent the input image by a set of subbands 
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consisting of the transform coefficients. Step 1011 is followed by bit 
allocation step 1102 and ROI coefficient scaling step 1003. At step 1102, 
the bits per coefficient (i.e., representation accuracy of the coefficient) 
assigned to the transform coefficients in each subband are determined. 
5 Equivalently, quantization step size for each subband may be determined. 
In case the allocated bits per coefficient or the quantization step size for 
each subband is predetermined, step 1102 is not performed. The 
allocated bits per coefficient is used at quantization step 1107. An 
alternative to performing the bit allocation in step 1102 based on the 
10 transform coefficients provided by step 1 101 is to perform the bit allocation 
based on the transform coefficients scaled up through step 1103. 

Before, after or together with steps 1101 and 1102, region of 
interest selection step 1104, ROI coefficient identification step 1105, and 
ROI coordinate description step 1106 are performed. At step 1104, a 
1 5 region of interest is selected on the input image and the coordinates of the 
selected region of interest are input to steps 1 105 and 1 106. At step 1 105, 
wavelet transform coefficients corresponding to the selected region of 
interest, i.e., ROI coefficients, in each subband are identified in order to 
emphasize the selected region of interest in the image by emphasizing the 
20 ROI coefficients in each subband containing the wavelet transform 
coefficients. The identification result of the ROI coefficients (i.e., categories 
of the coefficients), which depicts whether the transform coefficients 
correspond to each region of interest or regions outside of the region of 
interest, is input to step 1103. At ROI coordinate description step 1106, 
25 coordinates of the selected region of interest are encoded in order to 
effectively transmit or store the ROI coordinate information. 

At ROI coefficient scaling step 1103, the transform coefficients 
provided from step 1103, only the transform coefficients corresponding to 
the region of interest, are emphasized by multiplying the coefficient values 
30 with a scaling value which is assigned to the selected region of interest. 
According to the scaling, the transform coefficients corresponding to the 
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region of interest are to be quantized with a larger number of bits at 
quantization step 1107 and be represented more precisely than other 
transform coefficients. When a left bit shift value is used as a scaling 
value, the corresponding transform coefficients are emphasized by a left bit 
5 shift of the coefficient values. The identification result of ROI coefficients 
formed at the step 1105 is used to select which coefficients are to be 
scaled up. All the transform coefficients including the ones scaled up in 
step 1103 are given to quantization step 1107 to be quantized. The 
following steps are independent of which coefficients are scaled up. If a 
10 uniform quantizer is used and the scaling value to the region of interest is 
the quantization step size or the step size multiplied by integer value, ROI 
coefficient scaling can be done after the quantization step. In this situation, 
encoding method 1 100 is the same as the one in Fig. 1 . 

At step 1107, quantization is performed on the transform coefficients 
1 5 in each subband in order to represent the transform coefficients of each 
subband with quantization indices whose representation accuracy is 
specified by the allocated bit per coefficient or by the quantization step size 
for each subband. Through step 1107, quantization indices representing 
transform coefficients with a reduced or the same representation accuracy 
20 of the transform coefficient values are obtained. The obtained quantization 
indices are given to step 1108 entropy coding. 

At entropy coding step 1108, an entropy coding is performed on 
each bit element of the binary representation of the quantization indices in 
order to form an encoded data stream within which encoded bits generated 
25 from bit fields at a higher bit significance level of the quantization indices 
are placed in the earlier portion of the bit stream than other encoded bits 
generated from bit fields at a lower bit significance level. The encoding 
ordering of each bit field of transform coefficients in each subband is up to 
the encoding technique. The preferred encoding technique in step 1108 is 
30 either a bit plane coding such as binary arithmetic coding technique or a 
zero tree coding such as the SPIHT coding technique. 
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At transmitter step 1109, header bits such as size of the image, 
number of wavelet decomposition level, bit per coefficient or quantization 
step size for each subband, ROI description information, and priority or left 
bit shift value assigned to the region of interest are appended to the data 
5 bits formed by the entropy coder. The appended data is transmitted or 
stored as an encoding bit stream. 

When multiple regions of interest are to be emphasized, multiple 
regions of interest are identified at step 1104, ROI coefficients 
corresponding to each region of interest are identified and categorized into 
10 different categories at step 1 105, coordinates for each region of interest or 
for all the regions of interest are encoded at step 1 106. Steps 1 104, 1 105, 
and 1 106 for multiple regions of interest are the same as steps 104, 105, 
and 106 of the encoding method 100 (Fig. 1) for multiple regions, of 
interest. At step 1103, the values of the transform coefficients in each 
15 category can be scaled up by their own scaling value (multiplied by a 
scaling value assigned to each region of interest). As a result, each region 
of interest can be encoded with a different emphasis and the image quality 
of each region of interest can be separately controlled. 

In Fig. 12 a method of decompressing an image which is encoded 
20 by the encoding method in Fig. 1 1 with an emphasis on a selected region 
of interest is illustrated. An important difference between the decoding 
method in Fig. 12 and the decoding method in Fig. 2 is that the de- 
quantization is performed before the ROI coefficient de-scaling is 
performed in Fig. 12 while the de-quantization is performed after the ROI 
25 coefficient de-scaling in Fig. 2. The steps different from those in Fig.2 are a 
step for entropy decoding and a step for ROI coefficient de-scaling. The 
rest of the steps are the same. 

In Fig. 12, the decoding method 1200 comprises a receiver step 
1201 for receiving an encoded bit stream consisting of header bits and 
30 data bits. Step 1201 is followed by region coordinate decoding step 1202 
and entropy decoding step 1204. At step 1202, encoded data of the region 
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coordinates is decoded in order to obtain the coordinates of the region of 
interest (ROI) which is to be reconstructed with an emphasis in the 
reconstructed image. The decoded ROI coordinates are given to ROI 
coefficient identification step 1203 where wavelet transform coefficients 
5 corresponding to the region of interest are identified. The ROI coefficient 
identification result is used at ROI coefficients de-scaling step 1206. 

At entropy decoding step 1204, entropy decoding is performed on 
the incoming bit stream with the data bits which are received at receiver 
step 1201. The decoded bits are obtained for each bit field of the binary 
10 representation of the quantization indices of each transform coefficient. 
Within each quantization index, the bit field of the higher bit significance 
level is decoded earlier than the bit field of the lower bit significance level. 
An entropy decoder used in step 1204 is either a bit plane decoding 
technique or SPIHT decoding technique, if entropy encoding is done with a 
15 bit plane coding technique, decoding has to be done with a corresponding 
bit plane decoding technique; if entropy encoding is done with a SPIHT 
coding technique, decoding has to be done with a SPIHT decoding 
technique. Entropy decoded bits are organized as the bit field values of 
the binary representation of the quantization indices and are given to de- 

20 quantization step 1205. 

At de-quantization step 1205, de-quantization is performed on the 
quantization indices in order to obtain de-quantized values of the transform 
coefficients in each subband. The de-quantization scheme is specified by 
the quantization step size, bits per coefficient, or quantization table, either 
25 of which is taken out from the header bits. 

At step 1206, the values of the transform coefficients corresponding 
to a region of interest which is to be emphasized on the reconstructed 
image are scaled down. Scaling down is performed by dividing the values 
of the corresponding transform coefficients by a scaling value which is 
30 taken out from the header bits received at receiver step 1201. When the 
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scaling value is specified by a left bit shift value, the scaling down is done 
by performing a right bit shift on the coefficient values by the scaling value. 

At inverse wavelet transform step 1207, an inverse wavelet 
transform is performed on the transform coefficients in each subband in 
5 order to obtain a reconstructed image. The number of wavelet 
decomposition levels and the type of wavelet decomposition are provided 
by the header bits taken out from the encoded bit stream at step 1201. 
The number and the type have to be the same ones as used in the 
encoder. 

10 When multiple regions of interest are to be emphasized, based on 

header bits received from receiver step 1201, ROI coefficients 
corresponding to each region of interest are identified at the step 1203. 
Based on the ROI coefficient identification result and a priority or scaling 
value assigned to each region of interest provided by header bits, 
15 transform coefficients corresponding to each region of interest are scaled 
down according to a priority to each region of interest. 

Fig. 13 illustrates region of interest coding where encoding and 
decoding are done on a block by block basis. In this encoding and 
decoding scheme, an image is divided into a set of blocks (i.e., small parts 
20 of the image) whose shapes are rectangle, square, line, or combinations of 
some or all of the rectangle, square and line, and each block is treated as 
a separate input to the encoder or a separate output for the decoder. Each 
block which is outside the region of interest is encoded by an encoding 
method as in Fig. 1 or Fig. 11, without using ROI functionality, (without 
25 identifying ROI coefficients, without performing scaling up). An encoding 
tag which identifies a block as outside of the region of interest is added to 
the encoded bit stream for a block which is outside the region of interest. 
Each block which is inside the region of interest is also encoded without 
using ROI functionality, or alternatively the blocks inside the region of 
30 interest can be encoded by using ROI functionality such as ROI coefficient 
scaling. An encoding tag which identifies a block as inside the region of 
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interest is added to the encoded bit stream for a block which is inside the 
region of interest. Each block which overlaps the boundaries of the region 
of interest is encoded by an encoding method in Fig. 1 or Fig. 11 with ROI 
functionality. An encoding tag which identifies the region of interest 
5 boundaries is added to the encoded bit stream for a block which overlaps 
the boundaries of the region of interest. A bit rate for each block is 
allocated in such a way that the blocks within the region of interest are 
allocated the highest bit rate, the blocks overlapping the region of interest 
are allocated the next highest bit rate, and the blocks outside the region of 
10 interest are allocated the lowest bit rate. Alternatively, for the purpose of 
reducing the computational complexity at the encoding side and the 
decoding side, blocks which overlap the boundaries of the region of 
interest are regarded either as blocks which are inside the region of 
interest or the blocks which are outside the region of interest according to 
15 the number of pixels within the region of interest, percentage of pixels 
within the region of interest, or any other criteria. 

Each bit stream for each block can be aligned into an encoded bit 
stream for the entire image in the order that each block is encoded, or the 
bit streams for the blocks which are inside the region of interest can be 
20 aligned first, the bit streams for the blocks which overlap the boundaries of 
the region of interest can be aligned second, and the bit streams for the 
blocks which are outside the region of interest can be aligned last. In the 
latter case, the encoding tag for each block has to have location 
information specifying a location within the image. 
25 Based on an encoding tag assigned to bit portions for each block, 

decoding is done by a decoding method in Fig. 2 (if the encoding method 
in Fig. 1 is used), or by a decoding method in Fig. 12 (if encoding method 
in Fig. 11 is used). For bit portions preceded by a tag showing a block 
within or outside the region of interest, decoding is performed without using 
30 ROI functionality. If the encoding of the blocks within the region of interest 
is done by using ROI functionality such as ROI coefficient scaling, 
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decoding of the blocks within the region of interest needs to be done with 
ROI functionality. For bit portions preceded by a tag showing a block 
which overlaps the region of interest, decoding is performed with ROI 
functionality. 

5 By default, blocks within the region of interest are allocated the 

highest bit rate, blocks overlapping the region of interest are allocated the 
second highest bit rate, and blocks outside the region of interest are 
allocated the lowest bit rate at a given bit rate. However, the bit rate for 
each block can be controlled independently. If the decoding side wants to 

10 reconstruct regions which are not specified as the region of interest with 
higher fidelity, the blocks within the newly defined region are assigned a 
higher bit rate. 

When subband classification is used in the encoder as in Fig. 1 or in 
Fig. 11, each subband is classified into several sequences containing 

15 transform coefficients. Instead of performing ROI coefficient scaling in 
each subband, ROI coefficient scaling is performed on the quantization 
indices or on the transform coefficients in each sequence. Entropy coding 
is also performed on the scaled up result of each sequence. The encoding 
method can be the one in Fig. 1 , or the one in Fig. 1 1 if each sequence is 

20 assumed to be each subband in the encoding method of Figs. 1 or 11. 
Sequence based technique works with encoding method of Figs 1 and 11 
with every kinds of ROI coefficient scaling technique by treating each 
sequence as if it were each subband after each sequence is generated 
from subbands. 

25 When the encoded bit stream of the image is generated by en 

encoding method which uses a subband classification technique, entropy 
decoding and ROI coefficient de-scaling are done in order to obtain each 
sequence containing coefficients. Then several sequences originating 
from the same subband are de-classified into each subband so that the 

30 inverse wavelet transform can be performed. If each bit portion for each 
sequence is assumed to be a bit portion for each subband, the decoding is 
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done with a decoding method according to Figs. 2 or 12. As well as the 
encoding method, decoding method described in Figs. 2 or 12 works with 
subband classification technique by treating each sequence as if it were 
each subband in every step until the sequences are formed to subbands in 
5 order to perform inverse wavelet transform. 

When entropy coding is done on each block of coefficients in each 
subband or in each sequence in an encoding method as in Figs 1 or 11, 
ROI coefficient scaling method according to the Figs 3A - 6C and entropy 
coding method according to the Figs 7A - 8C are performed on each block 
10 of coefficients. To perform entropy coding on each block of coefficients, 
each subband or each sequence of coefficients is divided into blocks of 
coefficients. Each subband or sequence can be divided into blocks of the 
same shape or different shapes. The blocks can be of rectangular shape, 
square shape or line, or combinations of some or all of the rectangle, 
15 square, and line. In order to reduce the computational complexity for 
handling each block, all the subbands can have equal number of blocks of 
the same shape or alternatively equal size of blocks of the same shape. 
Each block is encoded separately. Blocks which are not corresponding to 
the region of interest are encoded without using the ROI coefficient scaling 
20 method. Encoding tag which shows the block is out of the region of interest 
is appended to the encoded bit stream of the block. Blocks which are 
corresponding to the region of interest are encoded by using the ROI 
coefficient scaling method. Encoding tag which shows the block is 
corresponding to the region of interest is added to the encoded bit stream 
25 of the block. The encoding tag may show that the all the coefficients in the 
block are corresponding to the region of interest or some of the coefficients 
in the block are corresponding to the region of interest. When all the 
coefficients in the block are corresponding to the region of interest, the 
encoding may be done without using the ROI coefficient scaling method. 
30 Alternatively, the blocks which have coefficients corresponding to 

the region of interest and coefficients corresponding to the regions out of 
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the region of interest can be classified either as the blocks whose 
coefficients are belonging solely to the region of interest or the blocks 
whose coefficients are belonging solely to the regions out of the region of 
interest. The classification is based on the number of coefficients 
5 corresponding to the region of interest, the ratio of the number of 
coefficients corresponding to the region of interest versus the number of 
the coefficients in each block, or any other criteria. Encoding for all the 
blocks is done without using the ROI coefficient scaling method. Instead, 
encoding tag which differentiates the encoded bit stream of each block 
10 either as a stream for the region of interest or a stream for the regions 
outside the region of interest is added to the encoded bit stream of each 
block so that the decoder can specify which bit portion is corresponding to 
the region of interest. 

When the entropy encoding is done on each block of coefficients 
15 within each subband or sequence, entropy decoding and ROI coefficient 
de-scaling method, which corresponds to the entropy encoding method 
and ROI coefficient scaling method used in the encoding side, as used in 
Figs 2 or 12, is done on each block. In this decoding situation, region of 
interest which is not specified at the encoding side or region of interest 
20 specified solely by the decoding side can be reconstructed with higher 
fidelity than the rest of the image or can be selectively reconstructed. 
Based on the newly specified region of interest, transform coefficients 
corresponding to the region of interest are specified according to the 
method in Figs 9. Blocks of coefficients which contain coefficients 
25 corresponding to the newly defined region of interest are reconstructed 
with higher bit rate than the other blocks in the subbands. Blocks of 
coefficients which contain only the coefficients corresponding to the region 
of interest will be allocated the highest bit rate. Blocks of coefficients which 
contain coefficients corresponding to the region of interest and those 
30 corresponding to the regions outside the region of interest will be allocated 
the same bit rate, the lower bit rate, or the lowest bit rate. Blocks of 
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coefficients which does not contain coefficients corresponding to the region 
of interest will be allocated the lowest bit rate. If 0 bit is allocated to a block, 
the block is not used to reconstruct the image. 

Bit allocation to each block can be more accurately performed 
5 based on the number of coefficients corresponding to the region of interest 
in each block. The larger the number of coefficients corresponding to the 
region of interest in a block, the block will be allocated the larger number of 
bit rate to reconstruct the image. The bit allocation can be done according 
to any criteria which reflects the degree of importance of each coefficient in 

10 reconstructing the region of interest or the entire image. For example, the 
degree of importance of each coefficient can be defined based on whether 
the coefficient is corresponding to the region of interest or not, how many 
pixels in the region of interest is represented by the coefficient, how may 
percentage of the pixels represented by the coefficient belongs to the 

15 region of interest. All or some of these criteria is used together with the 
number of coefficients corresponding to the region of interest within each 
block to determine the number of rate of bits to be allocated to the block of 
coefficients. 

Instead of allocating bit or bit-rate by using the criteria for each 
20 block, encoding or decoding ordering of the block of coefficients can be 
determined in such a way that the block which is to be allocated higher bit 
rate is given an earlier encoding or decoding ordering. Any other ordering 
can be defined based upon the criteria for each block. 

Fig. 14 is a flow chart of another method of encoding data in 
25 accordance with the present invention. The method begins with providing 
step 130 providing digital data corresponding to an image. A region of 
interest in the image is then selected by a user in step 132. However, the 
present invention also includes a method wherein an automatic target 
recognition system defines a region of interest based on a set of criteria. 
30 The automatic target recognition system may utilize a variety of 
approaches to identify the region of interest. For example, pattern 
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recognition software may be used to identify objects of interest in the 
image. After a region of interest has been selected in the image, a wavelet 
transform is then performed on the digital image data to obtain a set of 
subbands containing coefficients 134. As discussed above, a number of 
5 different types of wavelet transforms, such as mallet type transforms, spacl 
type transforms, packet type transforms, etc., may be used to obtain the 
coefficients. The coefficients corresponding to the selected region of 
interest are identified 136. The coefficients are then ordered so that at 
least one category of coefficients corresponds to digital image data 

10 representing the region of interest 138. The coefficients are then entropy 
encoded according to the category into which they were ordered 140. By 
ordering the coefficients into categories corresponding to the region of 
interest, it becomes possible to separately process and compress the 
different regions of the image. This is extremely valuable in situations 

15 where a large portion of the image consists of a relatively featureless 
background. 

There are several different approaches to determining the degree to 
which the region of interest will be compressed. One such approach is to 
let a user select the degree of compression. Another approach, shown in 

20 Fig. 15, is to determine at step 142 a total number of bits to be used to 
encode the digital data representing the image. Priorities are determined 
at step 144 for different regions of the image. Bits from the total number of 
bits are then allocated at steps 146 and 148 for encoding the various 
regions of the image according to the priority of the region such that high 

25 priority regions can be more accurately reconstructed than low priority 
regions. Typically, transmitting devices have a set bit rate at which they 
transmit a bit stream. In accordance with the invention, a portion of the bit 
stream to be transmitted is allocated at step 150 for transmitting the high 
priority regions of the image. For example, if the bit stream is transmitted 

30 at a rate of 1000 bits per second, 800 bits per second may be allocated to 
transmitting the encoded data representing the high priority regions of the 
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image and 200 bits per second may be allocated to transmitting the 
encoded data representing the low priority regions. Once a portion of the 
bit stream has been allocated at step 150 for transmitting the high priority 
encoded data, the bit stream is transmitted at step 152 to a remote 
5 location. Once all of the encoded data corresponding to the high priority 
regions of the image has been transmitted, the entire bit stream is 
allocated at step 154 to transmitting encoded data representing the lower 
priority regions of the image. 

Fig. 16 shows a block diagram of an embodiment of an encoding 

10 device of the present invention. Image data 600 is provided to a wavelet 
transformer 602. The wavelet transformer 602 performs a wavelet 
transform of the image data 600 to obtain subbands containing 
coefficients. A region of interest coefficient identifier 604 receives the 
region of interest coordinates and the subbands containing coefficients 

15 from the wavelet transformer 602 and identifies the coefficients 
corresponding to the region of interest. 

As discussed above, identification of the coefficients obtained from 
the wavelet transform that correspond to the region of interest is 
accomplished by tracing back the inverse wavelet transforms from the 

20 image domain to the wavelet domain. In this manner a set of wavelet 
transform coefficients that are used to reconstruct each pixel can 
separately be identified. Thus, the identification of the region of interest 
coefficients is independent of the shape of the region of interest and the 
region of interest can consist of disconnected regions. A subband 

25 classifier 608 classifies the coefficients and produces a classification map 
610. The classified coefficients are sent to a quantizer 612 and a rate 
allocation device 618. The rate allocation device 618 allocates a 
quantization step size to each class of coefficients and provides this rate to 
the quantizer 612. The rate allocation device 618 also produces a 

30 quantization table which includes information concerning the quantization 
step size allocated to each subband class. The quantizer 612 quantizes 
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the coefficients and provides the quantized coefficients to an entropy 
encoder 614. The entropy encoder 614 encodes the quantized coefficients 
according to received priority information concerning the region of interest 
616 provided to the entropy encoder 614. The output of the entropy 
5 encoder 614 is an encoded bit stream 622. 

Fig. 17 depicts a block diagram of an embodiment of a decoding 
device of the present invention. An encoded bit stream 622 and region of 
interest priority information 616 are received by an entropy decoder 624. 
The entropy decoder 624 produces a quantized output which is provided to 
10 an inverse quantizer 626. The inverse quantizer 626 also receives 
quantization information from a rate de-allocator 628 based on a 
quantization table 620. The inverse quantizer 626 produces coefficients 
that are sent to a subband declassifier 630 . The subband declassifier 630 
declassifies the coefficients according to a received classification map to 
15 produce a set of subbands which are provided to an inverse wavelet 
transformer 630. The inverse wavelet transformer 630 performs an inverse 
wavelet transform on the subbands to obtain image data 632 which can be 
used to reconstruct an image. Fig. 18 depicts an embodiment of the 
invention wherein the device includes a transmitting side 800 that encodes 
20 the image and transmits the encoded data to a receiving side 802 which 
displays the image. The transmitting side 800 has an encoder 804 with 
region of interest functionality. Initially, the encoder 804 encodes a low 
resolution, low fidelity or low bit rate version of the image and sends the 
encoded version to a transmitting device 806. The transmitting device 806 
25 transmits the encoded signal to a receiving device 808 located on the 
receiving side 802. The transmission may be accomplished across a wired 
transmission channel or wirelessly. The receiving device 808 on the 
receiving side 802 receives the encoded signal and provides it to a 
decoder 810 with region of interest functionality. The decoder 810 
30 decodes the image and sends it to a display 812 where it is displayed to a 
user. The receiving side 802 has a region of interest selector 814 which 
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allows a user to select a region of interest in the displayed image. The 
receiving side 802 also has a region of interest priority selector 816 which 
allows the user to select the priority with which the region of interest will be 
encoded. For example, the priority may be selected so that the region of 
5 interest will be reconstructed losslessly. Once a region of interest and a 
priority have been selected, the region of interest 814 and priority 816 
selector provide information concerning the selections to the decoder 810 
and to a transmitting device 818. The transmitting device feeds back this 
information to a receiving device 820 on the transmitting side 800. The 
10 receiving device 820 receives the information and provides it to the 
encoder 804. The encoder 804 then priority encodes the selected region 
of interest according to the information and then transmits the encoded 
region of interest information back to the receiving side 802 as previously 
discussed. The process can be repeated so that the user can redefine the 
15 region of interest based on an examination of the previous region of 
interest. Thus, the above discussed embodiment allows a user on the 
receiving side to determine the region of interest encoding interactively. 

Yet another application of the present invention involves integrating 
the region of interest concepts discussed above into a digital camera. In 
20 such an application, a region of interest and criteria for encoding the region 
is selected by a user of the digital camera on a view finder or display of the 
camera. The camera then records the information in the region of interest 
according to the criteria. By only recording a selected region of interest 
with a high resolution or fidelity, the amount of storage space required to 
25 store the digital image may be reduced. As the storage space in most 
digital cameras is very limited such an application is very beneficial. 

It is also appreciated that the above discussed encoding methods 
and apparatus can also be adapted to digitally reproduced motion pictures. 
For example, a selected portion of a bit stream being transmitting to allow 
30 for real time reproduction of an image could be dedicated to transmitting a 
region of interest in the resulting motion picture. Furthermore, the 
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dimensions and location of the region of interest could be constantly 
changing as the motion picture progresses. 

The above description of the invention is for illustrative purposes 
only. It should be understood that the selection and reconstruction of a 
5 region of interest according to the present invention can be utilized with 
other types of compression methods, and that the various means which 
are disclosed above have numerous equivalents which would be within the 
scope of knowledge of a person of skill in the art. The metes and bounds 
of the invention are defined in the appended claims. 
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WHAT IS CLAIMED IS: 

1. A method of image compression, said method comprising the 
steps of: 

providing digital image data including data on values and 
5 coordinates for a plurality of pixels; 

selecting a region of interest of an image represented by said digital " 
image data; 

sorting and prioritizing said digital image data according to at least 
two priority categories, with digital image data corresponding to the region 
10 of interest having a higher priority than digital image data corresponding to 
areas outside of the region of interest; and 

transmitting said sorted and prioritized digital image data to a 
remote location, with the digital information data corresponding to the 
region of interest being transmitted with higher priority than the areas 
1 5 outside of the region of interest. 

2. A method according to claim 1, wherein said sorting and 
prioritizing of said digital image data further comprises the steps of: 

performing a wavelet transform of all the pixel values in the digital 
image data to obtain transform coefficients; 
20 identifying the transform coefficients corresponding to the region of 

interest; 

emphasizing the transform coefficients corresponding to the region 
of interest by scaling up the transform coefficients corresponding to the 
region of interest; 

25 ordering the transform coefficients including the scaled up transform 

coefficients; and 

entropy encoding the transform coefficients to form a bit stream. 
3. A method of image compression as recited in claim 1, wherein 
said step of transmitting said sorted and prioritized digital image data to 
30 the remote location includes transmitting the sorted and prioritized digital 
image data corresponding to the region of interest at a higher rate such 
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image data corresponding to the region of interest at a higher rate such 
that the region of interest can be reconstructed at a higher fidelity than 
areas outside of the region of interest, said higher fidelity being provided by 
said sorting and prioritizing of said digital image data corresponding to the 

5 region of interest. 

4. A method for encoding and decoding an image, said method 
comprising the steps of: 

providing digital image data in a computer-readable format, said 
digital image data including data on values and coordinates for a plurality 

10 of pixels; 

sorting said digital image data according to a sorting protocol for the 
entire image, said digital image data being sorted and prioritized according 
to a predetermined prioritization formula; 

transmitting said sorted data to a receiver, and repeating said 
15 sorting and transmitting until a partial reconstructed image appears on a 
display at the receiver; 

selecting a region of interest based upon said partial reconstructed 

image; 

transmitting data from said receiver to a computer transmitting data 
20 identifying the selected region of interest; 

modifying the sorting of the digital image data based upon the 
selected region of interest, wherein digital image data corresponding to the 
region of interest is sorted and prioritized to have a higher priority than 
digital image data corresponding to areas outside of the region of interest; 
25 and 

transmitting said modified sorted and prioritized data to the receiver, 
with said region of interest being transmitted with higher priority than the 
areas outside of the region of interest. 

5. A method for encoding and decoding an image as recited in 
30 claim 4, wherein said step of transmitting said sorted data includes 
transmitting said sorted data onto a network, wherein said receiver is a 
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receiving computer on said network, and wherein said step of selecting the 
region of interest is performed at said receiving computer. 

6. A method of encoding digital data representing an image, the 
method comprising the steps of: 
5 providing digital image data in a computer-readable format, the 

digital image data including data on values and coordinates for at least one 
image; 

selecting at least one region of interest of an image represented by 
the digital image data; 
10 performing a wavelet transform selected from the group consisting 

of a mallet type wavelet transform, a spacl type wavelet transform, and a 
packet type wavelet transform on the digital image data to obtain subbands 
containing transform coefficients; 

identifying transform coefficients corresponding to the at least one 
15 selected region of interest; 

specifying a priority to each region of interest; 

classifying the transform coefficients in each subband into at least 
one sequence; 

allocating a bit-rate to each sequence of transform coefficients; 
20 quantizing the transform coefficients in each sequence by a 

quantization scheme selected based upon the allocated rate; 

scaling up bit significance levels of bit planes consisting of the 
quantized transform coefficients corresponding to each region of interest 
based upon the priority specified for each region of interest; 
25 modifying an encoding ordering of the bit planes consisting of 

quantized transform coefficients based upon the scaled up bit significance 
levels of the transform coefficients; and 

entropy encoding the bit planes of the quantized transform 
coefficients according to the modified encoding ordering to generate an 
30 encoded bit stream of digital image data. 
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7. The method of claim 6 further comprising the step of identifying 
the coefficients necessary and sufficient to encode the region of interest to 
be reconstructed up to a highest fidelity in a target spatial resolution image. 

8. The method of claim 6 wherein the step of identifying the 
5 coefficients corresponding to the region of interest further comprises the 

steps of: 

identifying pixels on boundaries of the region of interest; 

identifying pixels one pixel inside the region of interest along one of 
a vertical and horizontal direction of the image if at least one of a low pass 
10 and a high pass filter which perform the wavelet transform and an inverse 
wavelet transform has an odd filter length; 

tracing an input-output relationship of one of the wavelet transform 
and the inverse wavelet transform from an image domain to each subband 
domain to identify the transform coefficients which correspond to the pixels 
15 identified in the previous steps; 

forming boundaries of a corresponding region of interest in each 
subband domain based upon the identified coefficients in the previous 
step; and 

identifying coefficients surrounded by the boundaries in each 
20 subband as coefficients corresponding to the region of interest. 

9. The method of claim 6 wherein the step of identifying the 
coefficients corresponding to the region of interest further comprises the 
steps of: 

25 identifying a set of transform coefficients which correspond to each 

pixel in the region of interest by tracing an input-output relationship of one 
of a set of wavelet transforms performed on the pixel and a set of inverse 
wavelet transforms which reconstruct a pixel value; and 

forming a coefficient identification result such that a set of transform 

30 coefficients corresponding to each pixel can be identified. 
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10. The method of claim 6, wherein the step of entropy encoding 
the bit planes produces encoded information that is in the form of a number 
of bits, and the method further comprises the steps of: 

allocating the number of bits separately for use in representing the 
5 bit planes consisting only of the transform coefficients corresponding to the 
regions of interest in each sequence of transform coefficients, for use in 
representing the bit planes consisting of all the transform coefficients in the 
same sequence, and for use in representing the bit planes consisting of the 
transform coefficients corresponding to the regions outside of the region of 
10 interest in the same sequence such that the number of bits to reconstruct 
the region of interest and the number of bits to reconstruct the regions 
outside of the region of interest will be separately controlled; and 

aligning encoded bit portions of the bit planes such that a certain 
series of bit portions representing the region of interest will be transmitted 
15 at the earlier stage of transmission than bit portions representing regions 
outside of the region of interest. 

11 . The method according to claim 6 wherein the step of selecting a 
region of interest further comprises selecting a plurality of regions of 
interest in the image and assigning each region a priority and wherein the 

20 step of classifying the transform coefficients in each subband into at least 
one sequence further comprises the step of classifying the transform 
coefficients such that at least one sequence corresponds to each of the 
plurality of regions of interest. 

12. The method of claim 6 wherein the step of performing a wavelet 
25 transform on a digital image further comprises dividing the image into 

rectangular blocks of pixels such that some blocks only contain pixels 
which are inside the region of interest, some block only contain pixels 
which are outside of the region of interest, and some blocks contain some 
pixels which are inside the region of interest and some pixels which are 
30 outside of the region of interest and performing a wavelet transform on 
each rectangular block of pixels, and the method further comprises the 
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steps of 

defining the region of interest by selecting entire rectangular blocks 
of pixels to be included in the region of interest and by individually 
selecting pixels within rectangular blocks in which there exist boundaries 
5 between the region of interest and regions outside of the region of interest 
to be included in the region of interest; 

scaling up the bit significance level of all the transform coefficients 
corresponding to the rectangular blocks of pixels entirely within the region 
of interest and scaling up the bit significance level of the transform 
10 coefficients corresponding to pixels belonging to the region of interest in 
blocks that overlap the boundaries of the region of interest based upon the 
priority assigned to the region of interest; 

modifying a predetermined encoding ordering of the bit planes of the 
quantized transform coefficients in the blocks overlapping the boundaries 
15 of the region of interest based upon scaling up the bit significance level of 
the transform coefficients inside the region of interest; 

entropy encoding the bit planes of the quantized transform 
coefficients in the blocks which do not overlap the boundaries of the region 
of interest according to the predetermined encoding ordering, and 
20 encoding the bit plane of the quantized transform coefficients in the blocks 
which overlap the boundaries of the region of interest according to the 
modified encoding ordering; and 

adjusting the ordering of the encoded bit sequences of the bit 
planes of all the blocks such that the ordering of bit sequences of bit 
25 planes in the blocks inside the region of interest are given higher priority 
by a priority value assigned to the region of interest and the ordering of the 
bit sequences of the bit planes in the blocks outside of the region of 
interest and in the blocks overlapping the region of interest are the same 
as the encoding ordering of the corresponding blocks in order to generate 
30 the encoded bit stream of digital data. 
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13. The method of claim 6 wherein the step of entropy encoding 
further comprises performing data compression on the coefficients such 
that more information concerning regions outside of the region of interest is 
lost than is information concerning the region of interest. 
5 14. A method of decoding information representing an image, the 

method comprising the steps of: 

receiving the information in a computer-readable format, the 
received information including digital data representing an image and 
digital data concerning at least one region of interest in the image; 
10 locating at least one region of interest in the image to be 

reconstructed from the encoded bit stream according to the information 
concerning the at least one region of interest; 

identifying digital data corresponding to specified regions of interest; 
modifying a decoding ordering of the encoded bit stream based 
1 5 upon a priority assigned to each region of interest; 

entropy decoding the encoded bit stream according to the modified 
decoding ordering in order to obtain bit planes corresponding to quantized 
transform coefficients having a same bit significance level; 

scaling down the bit significance level of the bit planes including 
20 quantized transform coefficients corresponding to each region of interest in 
order to obtain original bit significance levels of the quantized transform 
coefficients, 

de-quantizing the quantized transform coefficients according to a 
de-quantization scheme to obtain sequences of transform coefficients; 
25 declassifying the sequences of transform coefficients into a set of 

subbands; and 

performing an inverse wavelet transform selected from a mallet type 
transform, a spacl type transform, and a packet type transform on the 
subbands in order to reconstruct the digital image data. 
30 -j 5 . The method of claim 14, further comprising the steps of: 
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identifying a number of bits separately for use in entropy decoding a 
portion of the encoded bit stream in order to obtain bit planes consisting 
only of transform coefficients corresponding to the region of interest, for 
use in obtaining bit planes consisting of all transform coefficients in the 

5 same sequence, and for use in obtaining the rest of the bit planes 
consisting of transform coefficients outside the region of interest in the 
same sequence such that the number of bits used to reconstruct the region 
of interest and the number of bits to reconstruct the regions outside of the 
region of interest will be separately controlled; and 

10 aligning the portions of received encoded bit stream such that a 

certain series of the bit portions representing the region of interest will be 
decoded at an earlier stage in reconstruction of the region of interest. 

16. The method of claim 14 wherein the step of receiving the 
information further comprises receiving the information such that 

15 information corresponding to the region of interest is received at a higher 
rate than information corresponding to regions of the image outside of the 

region of interest is received. 

17. The method according to claim 14 wherein the step of 
identifying digital data corresponding to a region of interest further 

20 comprises the steps of: 

identifying digital data corresponding to a plurality of regions of 
interest in the image represented by the information; 

determining a priority corresponding to each of the plurality of 

regions of interest; and 
25 reconstructing the image by reconstructing the plurality of regions of 

interest in a manner that is dependent upon each of the region's 

determined priority. 

18. The method of claim 14 wherein the encoded bit stream of the 
digital image data consists of sets of encoded bit planes representing 
30 rectangular blocks of pixels in the image, the method further comprising 
the steps of: 
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modifying a decoding ordering of encoded bit planes for rectangular 
blocks overlapping the boundaries of the region of interest based upon the 
priority value assigned to the region of interest; 

entropy decoding encoded bit planes for the rectangular blocks 
5 which do not overlap boundaries of the region of interest according to one 
decoding ordering, and entropy decoding encoded" bit planes for 
rectangular blocks which overlap the boundaries of the region of interest 
according to the modified decoding ordering in order to obtain bit planes 
consisting of a same bit significance level of the quantized transform 
10 coefficients in each sequence; 

scaling down the bit significance level of the bit planes consisting 
only of the quantized transform coefficients corresponding to the region of 
interest within the blocks overlapping the boundaries of the region of 
interest according to the priority value assigned to the region of interest in 
1 5 order to obtain an original bit significance level of the bit planes; and 

performing an inverse wavelet transform on each set of subbands 
containing quantized transform coefficients in order to reconstruct each 
rectangular block in the digital image. 

19. An encoding device for encoding information representing an 

20 image, the device comprising: 

receiving means for receiving information including data on values 
and coordinates for a plurality of pixels; 

selecting means for selecting at least one region of interest in the 
image represented by the information; 
25 wavelet based bit plane coder means for performing a wavelet 

transform selected from the group consisting of a mallet type wavelet 
transform, a spacl type wavelet transform, and a packet type wavelet 
transform on the information to obtain subbands containing coefficients; 

identification means for identifying coefficients corresponding to the 
30 at least one region of interest; 
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ordering means for ordering the coefficients according to a plurality 
of categories wherein at least one category corresponds to coefficients 
representing the region of interest and at least one category corresponds 
to coefficients representing regions outside of the region of interest; 
5 entropy encoding means for encoding the coefficients according to 

the category into which the coefficients were placed to obtain a number of 
bits representing the encoded coefficients; and 

transmitting means for transmitting the bits in a bit stream to a 
remote location, such that the manner in which the bits are transmitted 
1 0 depends upon the category of coefficients to which the bits correspond. 
20. The encoding device of claim 19 further comprising: 
bit allocation means for allocating a predetermined number of bits 
for use in representing each category of coefficients wherein the number of 
bits allocated to represent the categories of coefficients corresponding to 
1 5 the region of interest of the image is such that the region of interest of the 
image can be reconstructed with a higher fidelity than the regions of the 
image outside of the region of interest; and 

bit stream allocation means for allocating a fixed portion of bits of 
the bit stream transmitted per unit time for transmitting bits corresponding 
20 to categories of coefficients corresponding to the region of interest. 

21. The encoding device of claim 19 wherein the selecting means 
select a plurality of regions of interest further comprising priority assigning 
means for assigning priorities corresponding to each of the plurality of 
regions of interest and wherein the entropy encoding means further 
25 comprise priority encoding means for encoding the categories of 
coefficients in a manner that depends upon the priorities assigned to the 
plurality of regions of interest in the image to which the category of 

coefficients corresponds. 

22. The encoding device of claim 19 wherein the ordering means 
30 further comprise block ordering means for ordering the coefficients 
according to categories such that some categories of coefficients contain 
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information that corresponds to rectangular blocks of pixels in the image 
wherein all the pixels in the corresponding block are either inside the 
region of interest or outside of the region of interest, and some categories 
of coefficients contain information that corresponds to rectangular blocks of 
pixels in the image wherein some of the pixels in the corresponding block 
Ve inside the region of interest and some" of the pixels are outside of the 

region of interests. 

23. A decoding device for receiving and decoding information 

representing an image, the device comprising: 

receiving means for receiving the information including data on 
values and coordinates for a plurality of pixels; 

identification means for receiving the information from the receiving 
means and identifying at least two categories of information corresponding 
to regions of the image such that at least one category corresponds to 
15 information representing a region of interest in the image and at least one 
category corresponds to information representing a region in the image 
outside of the region of interest; 

entropy decoding means for entropy decoding the information 
according to the information's category; 
20 reconstructing means for reconstructing the image from the entropy 

decoded information such that the manner in which regions of the image 
are reconstructed depends upon the category of the information 

representing the region. 

24. The decoding device of claim 23, wherein the received 
25 information is in the form of a bit stream and a fixed portion of bits in the bit 
stream received per unit time corresponds to the region of interest, and 
wherein the reconstructing means reconstruct the region of interest more 
quickly than regions of the image outside of the region of interest and with 
a higher fidelity than regions outside of the region of interest. 
30 25. The decoding device of claim 23 wherein the receiving means 

receive the information corresponding to the region of interest before 
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receiving the information corresponding to regions of the image outside of 

the region of interest. 

26. The decoding device of claim 23 wherein the identification 

means further comprise: 
5 multiple region of interest identification means for identifying a 

plurality of categories corresponding to a plurality of regions of interest in 
the image represented by the information; and 

priority determining means for determining a priority corresponding 
to each of the plurality of regions of interest; 
10 and wherein the reconstructing means reconstruct the plurality of 

regions of interest in a manner that is dependent upon each of the region's 
priority. 

27. The decoding device of claim 23 wherein the identification 
means further comprise rectangular block identification means for 

15 identifying categories of information containing information that 
corresponds to rectangular blocks of pixels in the image wherein all the 
pixels in each corresponding block are either inside the region of interest or 
outside of the region of interest and categories of information containing 
information that corresponds to rectangular blocks of pixels in the image 

20 wherein some of the pixels in each corresponding block are inside the 
region of interest and some of the pixels are outside of the region of 
interests. 

28. A method of encoding digital image data representing an image, 
said method comprising the steps of. 
25 selecting at least one region of interest in an image represented by 

said digital image data; 

performing a wavelet transform on the digital image data to obtain 

transform coefficients; 

identifying the transform coefficients corresponding to the region of 

30 interest; and 

assigning a priority to the at least one region of interest. 
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29. The method according to claim 28, further comprising the steps 

of: 

emphasizing the transform coefficients corresponding to the region 
of interest by scaling up the transform coefficients corresponding to the 

5 region of interest; 

ordering the transform coefficients including the scaled up transform 

coefficients; 

quantizing the transform coefficients including the scaled up 

transform coefficients; and 
1 o entropy encoding the transform coefficients to form a bit stream. 

30. The method according to claim 28, further comprising the steps 

of: 

quantizing the transform coefficients to obtain quantization indices; 
emphasizing quantization indices corresponding to the region of 
1 5 interest by scaling up the quantization indices corresponding to the region 
of interest; 

ordering the quantization indices including the scaled up 

quantization indices; and 

entropy encoding the quantization indices to form a bit stream. 
20 31 . a digital camera for producing digital image data representing 

an image, the digital camera comprising: 

digital image data producing means for producing digital image data 

representing an image; 

selecting means for selecting a region of interest in the image 

25 represented by the digital image data; 

wavelet transform means for performing a wavelet transform on the 
digital image data to obtain transform coefficients; 

identification means for identifying transform coefficients 
corresponding to the at least one region of interest; 
30 ordering means for ordering the transform coefficients according to 

a plurality of categories wherein at least one category corresponds to 
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coefficients representing the region of interest and at least one category 
corresponds to coefficients representing regions outside of the region of 
interest; 

encoding means for encoding the transform coefficients so that the 
transform coefficients corresponding to the region of interest are encoded 
in a manner that allows the region of interest in the image to be 
reconstructed more accurately than regions of the image outside of the 

region of interest; and 

storage means for storing the encoded transform coefficients. 



BNSDOCID: <WO 9949412A1J.> 



WO 99/49412 



PCT/US98/19065 



1/34 



100-^- 
IMAGE- 



ENCODING METHOD 



101 



WAVELET 
TRANSFORM 



102 

S 



BIT 
ALLOCATION 



1 



REGION 
SELECTOR 



104 



ROT COEFFICIENT 
IDENTIFICATION 



103 



QUANTIZATION 



T 
105 



i 



106 



ROI COORDINATE 
DESCRIPTION 



ROI COEFFICIENT 
SCALING 



107 



ENTROPY 
CODING 



108 



109 



TRANSMITTER 



I I 



HEADER 


DATA 


BITS 


BITS 



Fig. 1 



BNSDOCID: <WO 9949412A1J_> 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/19065 



2/34 



DECODING METHOD 200 



Fig. 2 



HEADER 
BITS 


DATA 
BITS 


\ 


t 





-201 



RECEIVER 



202 

f 



REGION 
COORDINATE 
DECODING 



203 

\ 



ROl COEFFICIENT 
IDENTIFICATION 



EN TROPY 
DECODING 



204 



205 



ROl 


COEFFICIENTS 


DE- 


SCALING 



I 



DE- QUANTIZATION 



206 



INVERSE 
WAVELET 
TRANSFORM 



I 



207 



RECONSTRUCTED 
IMAGE 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCID. <WO . 99494 12A1 I > 



WO 99/49412 



PCT/US98/19065 



BIT 

SIGNIFICANCE 
LEVEL 



3/34 



Fig. 3 A 



N-l 
N-2 



I 

O 



Z(O) 



BIT 

SIGNIFICANCE 
LEVEL * 



N+S- I 

i 
i 
I 
i 

N 

N-l 

i 
i 
i 



S- I 
\ 
I 
i 

o 



~ SCALING UP THE MAGNITUDE OF THE 
QUANTIZATION INDICES 



bn(M )\n-h bit 

FIELD IN 
QUANTIZATION 

INDEX 'M' 

D : 0, /, • • •, N-l 



b TJ, 


i 






bN-l 
(je ) 


bN-l 
( ) 


bN-2 
(0) 


1 
i 


bN-2 
(Js) 




bN-2 
(Je) 


bN-2 
( ) 


I 
t 
1 


1 — 

1 
i 


i 
i 




i 
i 

— i — 




1 
1 

1 


i 
i 

1 — 


i 
i 
i 




i 
i 




bi(0) 


1 
i 


bi(js) 




bt(je) 


bi( ) 


bO(0) 


i 
i 


bo(js) 




bo(je) 


bo( ) 



Zljs) 
\ 



Z (je) 

s 



QUANTIZATION 
INDICES FOR THE 
REGION OF INTEREST 



QUANTIZATION 
INDEX 




SCALE UP ( LEFT BIT SHIFT 
= s ; 0<S = /V 





bN-l 

(JS) 




bNr 1 . 
(je) 




i 






bNrS 
(JS) 




bNrS 
(je) 


bN-l 

(0) 




bNyS-l 
(JS) 




bNrS-l 
(je ) 


bN-i 
( ) 


1 

I 
1 










i 
■ 


b S(0) 




b 0(j S ) 




^(je) 


bs( ) 


bs-i 
(0) 




1 i 
1 | 
1 1 
1 | 
1 | 
1 1 


bs-i 
( ) 


1 

1 

1 






bo(0) 




bo( ) 



Z(0 ) 



Z( js ) 



Zl je) 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCID: <WO 9949412A1_I_> 



WO 99/49412 



PCT/US98/19065 



4/34 



BIT 
FIELD 



bN-l 
bN-2 



bl 
bo 



BIT 
FIELD 



bN-l 
bN-2 

bN-S 
bo 



~ SCALING UP THE BIT 
SIGNIFICANCE OF THE 
QUANTIZATION INDICES 



AM 




N-l 1 


i 
i 


N- 1 




N-2 




N-2 


i 

— i — 


N-2 


N-2 


i 
i 




i 
i 


i 


l 
i 


i 
i 


i 




- — r — 
i 
i 


— i — 
i 

— i — 


i 
l 


i 
i 


/ 




/ 


i 


/ 


/ 


0 




0 


i 
i 


0 


0 



Z(o) 



Fig. 3B 



BIT SIGNIFICANCE 
LEVEL ■■ 2 N-l 



Z(js) Z(je) 

v ^ — ' 

QUANTIZATION INDICES FOR 
THE REGION OF INTEREST 



SCALE UP (LEFT BIT SHIFT = S) 
0<S = N 

BIT SIGNIFICANCE LEVEL - 
2 N+S-I = 2 N-I x 2 S 



N-l 








• 


N+S-l 


N-l 


N-2 






N+S-2 






N-2 


i 
i 
1 






t 
I 
l 




1 
I 

1 1 


1 

i 
I 
I 


N-S 






N 




N 


N-S 


i 
I 

[ i 






1 
i 




1 
i 


1 
1 


0 






S 


i 


S 


0 



Z(o) ZCjs) Z(je) 

v ^ y 

QUANTIZATION INDICES FOR 
THE REGION OF INTEREST 



SUBSTITUTE SHEET (RULE 26) 

BNSOOCID:<WO . 994S412A1_I_> 



WO 99/49412 



PCT/US98/19065 



5/34 

Fig.3C 



bN- i 


0 


0 


0 




0 


0 


bN- 2 


1 


1 


1 




1 


1 




i 
i 














1 
J 
1 












bi 


1 
i 
» 












bo 


N-l 




N-l 




N-l 


N-l 



Z(o) z(js) Z(je) 

V ^ s 

QUANTIZATION INDICES 

FOR THE REGION OF INTEREST 




SCALE UP ( LEFT BUT 

SHIFT= S ) 0 <S= N 



bN-t 5 SO O OS 
bN-2 s + i s+/ i i i 5+/ 



bN-S-l 2S 2S S S S 2S 



bO N+S+l N+S+l N-l N-l N-l A/+S+/ 

1 1 1 1 1 ' 3— 

Z(o) Z(js) Z(je) 

\ ^ / 

OUAN TIZATION INDICES 
FOR THE REGION OF INTEREST 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCID: <WO 9949412A1 I. > 



WO 99/49412 



PCT/US98/19065 



6/34 



QUANTIZATION INDEX FOR 
COEFFICIENT, Z(j) 




S: LEFT SHIFT 
VALUE TO THE 
ROI 



COEFFICIENT 



GO TO THE NEXT 
QUANTIZATION INDEX 




Fig. 3D 



BNSDOCIO: <WO 99494 12A1 J > 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/19065 



7/34 



QUANTIZATION INDEX FOR 
COEFFICIENT, Z(j) 



{ s ° Fig. 3E 




YES 



SCALE UP THE BIT SIGNIFICANCE 
OF EACH BIT FIELD 

* \ FOR(k = N-ij k^O; k — ) \ 

. I , 

(BIT SIGNIFICANCE OF \ 

Kbit field, bk(j)+ s s ) 



S : LEFT BIT 
SHIFT TO 
THE ROI 

COEFFICIENT 



1 



SUBSTITUTE SHEET (RULE 26) 

BNSDOCID: <WO 9949412A1J_> 



WO 99/49412 



PCT/US98/1906S 



8/34 

Fig. 3F 





QUANTIZATION INDEX FOR 




COEFFICIENT, Zip 



}=0 




YES 



t 

RE- ASSIGN THE ENCODING ORDERING 
OF EACH BIT FIELD 



■^-( ^FOR ( k = N-l ; k= Q ,k— ) ) 

V 



( ENCODING ORDERING ASSIGNED \ 
[ TO EACH BIT FIELD, bk(j ) J 



- = S 



S - LEFT BIT SHIFT 
TO THE ROI 
COEFFICIENT 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCIO: <WO 9949412A1 I > 



WO 99/49412 



9/34 



PCT/US98/19065 



BIT 

SIGNIFICANCE 
LEVEL A 



Fig. 4A 



ROI COEFFICIENT SCALE UP FOR 
MULTIPLE REGIONS OF INTEREST 



N-l 


bN-l(0) 




bN-i 

dsn 


i 
i 


bN-i 

(}*n 




bN-i 
(Js2) 


bN-i 
(Je2) 






N-2 


bN-2 
(0) 




bN-2- 

(is i ) 


i 


bN-2 

()ei) 




bN-2 
(JS2) 


bN-2 
(je2) 








1 
i 

1 




1 
1 








i 










i 
i 
i 




1 
1 

1 
















1 


t>l(0) 




bi(jsl) 
















0 


bo(o) 




bo 

(jsl) 


1 
i 


bo 
(jel) 




bo 

(JS2 ) 


bo 

(je2) 







Z(O) 



BIT 

SIGNIFICANCE 
LEVEL 



Z(jsl) 
\ 



Zijel) 



QUANTIZATION 
INDICES FOR THE ROI 1 



A/+S/- / 
N + S2-I 



N-l 
N-2 



I 

0 



Z(js2) Z(je2) , 

V y QUANTIZATION 

' I NDEX 

QUANTIZATION 

INDICES FOR ROI 2 



SCALE UP 
LEFT SHIFT S f FOR ROI 





bN-i , 

(JSI) 






\ 


LEFT SHIFT S2 FOR 










(]S2 ) 






bN-Sl 

dsn' 






bN-S2 
(JS2) 




bN-i 
10) 




bN-Sf- 

(jsl) 


1 




bN-l 

{ ) 


bN-S2- 
(js2l 




bN-i 

( ) 












1 






1 

1 

1 






bo , , 
(jsl ) 












1 
1 

— 1 — 










bo 

(JS2) 




1 

— 1 — 






— 1 — 




1 

1 


t>o(0) 




bo( ) 


bo( ) 



I) 



2(0) 



Z(jsl) 
\ 



Zijel) 2(js2) 2(je2) 

' V ^ ' QUANTIZATION 

ROI I ROI 2 ,NDEX 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCID:<WO . 9949412A1 I > 



WO 99/49412 



PCT/US98/19065 



N-Si-I 



bN-i ( jsl) 



N-S2-I 



N-l 
N-2 



4 



10/34 

bN- i(jel) 

4- 



Fig. 4B 



bN-i(js2) bN-itjez) 
A r—f 



bo (jsl) bo(jel) 



bN-i (0) I 
1 



/ 

0 











— 1— 




1 




— i — 
1 




b 0 (0) 





bN-i( ) 



bo( ) 



-4- 

b 0 (jS2) \ 

bo(je2) 



bN-IO 



bo( ) 



Z(jsl) Z(jel) Z(js2) Z(je2) quantization 
^ „ s V y INDEX 



ROI 1 



ROI 2 



PRIORITY TO THE ROI I AND ROI 2 ARE WITHIN 
THE FOLLOWING RANGE (SUPPOSE S2 = S t ): 

S2 ^ Si ^ S2 
0 <Z S 2 =N 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCID: <WO_9949412A1_l_> 



WO 99/49412 



PCTAJS98/19065 



BIT 

SIGNIFICANCE 
LEVEL 



11/34 




Fig. 5A 



P LEVELS 



N-P LEVELS 



Z(js) Z(je) 

v ^ v 

ROI COEFFICIENT 



BIT 

SIGNIFICANCE 
LEVEL 



N-l 
N-P 



SCALE UP OCCURS AFTER P 
STAGES OR PATHS 
ARE COMPLETED. 




ZHo) Zl(js) Zl(je) 



Z2(o) Z2(js) Z2(je) 



BNSDOCID: < WO „ 99494 12A1 I > 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 PCT/US98/19065 



12/34 



BIT 
FIELD 

bN-l 



bN-P 



bi 

bo 



N-l 




N-l 




N-l 


N-l 


■ i 




1 
1 




1 
1 




N-P 




N-P 




N-P 


N-P 


1 
1 
i 












i 

/ 




1 




1 


1 


0 




0 




0 


0 



Fig. 5B 



Z(o) Z(js) Z(je) 

ROI COEFFICIENT 



\7 



SCALE UP OCCUR AFTER 
P STAGES OR PATHS 
ARE COMPLETED. 



BIT 
FIELD 

bN-l 



bN-P 

i 
i 
i 
i 



b, 
bo 



N-l 




N-l 








— h- 
i 












N-P 




N-P 









BIT 
FIELD 



b N-P-l 



bo 



ZKO) Zl(js) Zl(je) 



N-P-l 



V 7 7 
N-P+S 

/ / / 



'/// 




A 



N-P-l 



Z2(o) Z2(js) Z2(je) 



BNSDOCID: <WO 99494 12A 1.1. > 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCI7US98/19065 



BIT 

FIELD 
bN-l 



bN-P 



b, 
bo 



P-/ 



N- 1 



Z(O) 



P-l 




13/34 



p-l 



N-l 




Z(je) 



ROI COEFFICIENT 



Fig.5C 




SCALE UP OCCUR AFTER P STAGES 
OR PATHS ARE COMPLETED 



BIT 
FIELD 



bN-l 

bN-p 
bN-p-i 

bo 



P-l 



P-l 



P-l 



+ 



Zl( ) Zl(js) Zl(je) 



P+S 



N+S-i 



Y / 



7/ 
N-b 



P+S 



N+S-l 



Z2(o) Z2(js) Z2(je) 



BNSDCXID: <WO 99494 12A1J. > 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/1906S 



14/34 



lu 
U_ 

to 

CD 
CL 

2cT 



tfc 

fcf 

CVJ 
CO 

CD Lu 

CO 



Q 
Lu 
—J 

o 

CO + 



a. 



— c\j 



o 
ct 



o 
cr 



CO 




CM 
CL 
i 



CL CL 



a. 




CM 

•—4 

o 
ct 



o 

Cc 



I 

CL 

£■ i 



co 




CL 
i 



LU 

O 

^ — 1 
<l Lu 



S2 

CO 



CL 
i 




SUBSTITUTE SHEET (RULE 26) 



BNSOOCID: <WO 9949412A1 I. > 



WO 99/49412 



PCI7US98/19065 



15/34 



ROI 2 

CAN BE 

SCALED 
UP 




Fig. 6B 



N-Pl 



^2 



s / 



P2 



+ 



A 



WZ3 



\L 



N-P\-P2 



ROI I ROI 2 



ROI 2 IS SCALED 
UP BY S2 



'ZZZ 



^S2 



N-P1-P2 



ROI1 ROI 2 



994941 2A1 I > 



SUBSTITUTE SHEET (RULE 26) 



i 



WO 99/49412 



PCT/US98/19065 



Fig.6C 



16/34 

ROI 1 IS SCALED UP BY S/ AFTER\ 
PI STAGES. \ 

ROI 2 IS SCALED UP BY S2 AFTER I 
P/+P2 STAGES J 



X 

L 



PI 



N-PI 



+ 




j L 



ROI 1 



+ 

A 



TZZT 



ROI I 



—i 1 lr 

ROI 2 



N-PI-P2 



BNSDOCID: <WO_.9949412A1_l .> 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 PCT/US98/19065 



BIT 

SIGNIFICANCE 
LEVEL 



W-/+S 

t 

N 
N-l 



S 

b 



17/ 34 

Fig. 7A 

















\ 
S 

f 




























































! 1 

i 1 

i i 
i 1 
i l 











DEFINING 
BIT 
PLANES 



LZJ. 




BIT PLANE 1 
-BIT PLANES 



-BIT 
PLANE 
S+/ 



i t i r 



. BIT 
PLANE N 



— BIT 
PLANE 
N+ I 

BIT 
PLANE 
W+S 



ROI COEFFICIENT 



BIT PLANE 1 





BIT PLANE CODER 




BIT STREAM 1 







BIT PLANE N 

I 

I 
i 
i 



BIT PLANE N-t-S 



BIT PLANE CODER 



BIT STREAM N 



BIT PLANE CODER 



1 



ROI IS 
FULLY 
ENCODED. 



BIT STREAM N+S 



ENTIRE 
IMAGE 
IS FULLY 
ENCODED 



BNSDOCID: <WO... 99494 12A1. I_> 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/19065 



18/34 Fig.7B 

■BIT STREAM OF EACH SUBBAND 





TAG 1 


BIT STREAM 1 




TAG S 


BIT STREAM S 







TAGN BIT STREAM N 



(I) 



(2) 




TAG N+l BIT STREAM N+l 



TAG N+S BIT STREAM N + S 



(3) 



r \) BIT STREAM I ~ BIT STREAM S - REPRESENTING ROI, 

2) BIT STREAM S- I— BIT STREAM N - REPRESENTING ROI- THE REST 
OF THE IMAGE, 

3) BIT STREAM W-/~S/T STREAM N- S REPRESENTING THE REST 
OF THE IMAGE. 



B IT STREAM 1 



BIT RATE 
CONTROLLER 
FOR ROI 



BIT STREAMS 



BIT STREAMS*! 



BIT STREAM N 



BIT STREAM /' 



RATE 
SPENT 



BIT STREAM S' 



BIT RATE 
CONTROLLER 
FOR ROI + 
THE REST OF 
THE IMAGE 

F 



BIT STREAM A/+/ 



BIT STREAM N+S 



BIT STREAM S + /' 



Bl T STREAM N 



RATE SPENT 



BIT RATE 
FOR REST 
OF THE 
IMAGE 



BIT STREAM A/+/' 



BIT STREAM A/+S' 



RATE SPENT 

• BIT RATE CONTROL FOR EACH CATEGORY OF INFORMATION 



v 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCID: <WO.._ 9949412A1. 1.> 



WO 99/49412 PCT/US98/19065 



19/34 

Fig. 7C 



2N-I 
2N-2 



N 
N- I 

N-2 



I I 



I t I i ~ 



DEFINING 
BIT PLANES 



c 



a 



C5 



Cm 



Cm 



Zljs) Z(je) 



ROI COEFFICIENT 
BIT PLANE 1 - 



i 
i 

i 

BIT PLANE N 

i 
i 
i 



BIT PLANE 1 
BIT PLANE 2 

i 

I 



BIT PLANE N 

| I ) — BIT PLANE N+l 



| \y ~BIT PLANE A/+2 



3=D 



I I ) — BIT PLANE 2N 



BIT PLANE CODER 




BIT STREAM 1 










• BIT PLANE CODER 




BIT STREAM N 



ROI IS 

FULLY 

ENCODED 



♦ 



BIT PLANE 2N- 



BIT PLANE CODER 



BIT STREAM 2N 



ENTIRE 
J IMAGE 
IS 

FULLY 
ENCODED 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCID: <WO. 9949412A1..I. > 



WO 99/49412 



20/34 



PCT/US98/19065 



BIT STREAM OF EACH SUBBAND 



h- CD 


3» 


TAG 1 BIT STREAM 




TAG N 


BIT STREAM N 





en 



TAG 2N\ BIT STREAM 2N \ 



®l 



(7) B/T STREAMS BIT STREAM N *• REPRESENTING ROI 

(3) B/T STREAM N+1^ BIT STREAM 2 A/ : REPRESENTING 
THE REST OF THE IMAGE 



BIT STREAM 1 



BIT STREAM N 



BIT STREAM N- 1 



BIT STREAM 2N 



BIT RATE 
CONTROLLER 
FOR RO I 



BIT STREAM 1' 



BIT STREAM N' 



RATE SPENT 



BIT RATE 
CONTROLLER 
FOR ROI 



BIT STREAM N- /' 



BIT STREAM 2N' 



RATE SPENT 

BIT RATE CONTROL FOR EACH CATEGORY OF INFORMATION 



Fi g. 7D 



BNSDOCID: <WO 9949412A1J^> 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/19065 



21/34 



Uj 

<3 
— I 
CL 

h~ 

CO 

>~ 
QO 

Q 

8 

>- 
CL 
O 
Or 

Uj 



1 

Q 
O 
O 

5 
fc 

Ul 

—J 

§ 



Uj 

a: 



o 

E 



oo 



CL — 
CD CD 



+ 
CL 

Uj 
Z 

a! 



i «£ 

Uj 

a! 



CO CD 



Uj Uj 

CL CL 
CD CD 



Uj 

a! 

CO 



+ 
Uj 
— > 



D 0 



2 Uj 



u.cd3 U 

UJ Q. 



CO 



.Si,uj 

isj ^ 

u_ 

- u. 

Uj 

O 
O 



rsi 

V 



o 

CC 



— CL 





I 




CO 




+ 


Uj 


CL 


o 


1 






<5 — 1 




. OUj 




CD — Uj 




2~J 




e> 




CO 





I 

CL 
I 



SUBSTITUTE SHEET (RULE 26) 



BNSOOCID: <WO.._ 9949412A1 l_> 



WO 99/49412 



22/34 



PCT/US98/19065 



Fig. 8B 



1) BIT PLANE 1 

2) ! 



P) BIT PLANE P 



P+l) BIT PLANE P+ / 



BIT PLANE CODER 



BIT PLANE CODER 



BIT PLANE CODER 



BIT STREAM 1 



BIT STREAM 



* ,i 

I AM P | 



ROI CODING 
NOT YET 
STARTED 



BIT STREAM P+ I 



ROI 
; CODING 
STARTS. 



P+S J BIT PLANE P+S 



BIT PLANE CODER 



BIT STREAM P- S 



N) BIT PLANE N 



N + S) BIT PLANE N+S 



BIT PLANE CODER 



BIT PLANE CODER 



BIT STREAM N 




BIT STREAM W+ 



ROI IS 
FULLY 
I ENCODED 

ENTIRE 
IMAGE IS 
ENCODED 



TAG I 



BIT STREAM 1 



TAG 2 BIT STREAM 2 



TAG P 



BIT STREAM P 



Y 



G 



TAG P+ / 



B I T STREAM P+ / 



TAG P+ S 



BIT STREAM P+S 



G 



TAGP+S+I BIT STREAM P+S+l 



TAG N BIT STREAM N 



G 




TAG W+/ 



BIT STREAM N+ I 



TAG W+S 



BIT STREAM N+S 



BIT STREAM OFTHESUBBAND 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCIDkWO 99494 12A1 I > 



WO 99/49412 



PCT/US98/19065 



23/34 



Fig.8C 



I ) BIT STREAM l~P: REPRESENTING THE ENTIRE IMAGE 

I BEFORE ROl CODING OCCURS) , 

2) BIT STREAM P* /~P+S • REPRESENTING ROl , 

3) BIT STREAM P+S+ / ~N- REPRESENTING ROl + THE REST OF THE IMAGE 

4) BIT STREAM N+ l~N-+ S- REPRESENTING THE REST OF THE IMAGE 



BIT STREAM I 



BIT STREAM P 



BIT RATE 
CONTROLLER 
FOR THE IMAGE 



BIT STREAM P+ / 



BIT STREAM P+S 



BIT STREAM ? \ 



BIT STREAM P' 



RATE SPENT 



BIT RATE 
CONTROLLER 
FOR ROl 



BIT STREAM P+ I' 



BIT STREAM Ps-5 / | 



RATE SPENT 



BIT STREAM PH-S+/ 



BIT STREAM N 



JL 



BIT RATE 
CONTROLLER 
FOR ROl + 
REST OF THE 
IMAGE 



B IT STREAM A/+ / j *- BIT RATE 

--—-—-——^ < CONTROLLER 

FOR THE REST 
OF THE IMAGE 



BIT STREAM N+S 



BIT STREAM P+S+l' 



BIT STREAM N' \ 



RATE SPENT 



T 



BIT STREAM N+l' 



BIT STREAM N+S' 



RATE SPENT 

BIT RATE CONTROL FOR EACH CATEGORY OF 
INFORMATION 



SUBSTITUTE SHEET (RULE 26) 



BNSDOCID: <WO . 994941 2A I I > 



WO 99/49412 



PCT/US98/19065 



24/34 



Fig. 9 



ROI COEFFICIENT IDENTIFICATION 




REGION OF INTEREST 



# IMAGE AND SELECTED REGION OF INTEREST 

O ; PIXELS ON THE BOUNDARIES OF REGION OF INTEREST 

• • PIXELS ON WITHIN 1 PIXEL INSIDE THE 
' THE BOUNDARIES 



TRANSFORM COEFFICIENTS 

CORRESPONDING TO THE SELECTED 
REGION OF INTEREST. 




ROI COEFFICIENT IDENTIFICATION IN 

1 LEVEL WAVELET DECOMPOSED SUBBANDS 



BNSDOCID: <WO 994941 2A1_ I > 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/19065 



25/34 




BNSDOCID: <WO 9949412A1..I _> 



SUBSTITUTE SHEET (RULE 26) 




BNSDOCID: <WO . . 9949412A1 .1 > 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/19065 



27/34 



Fig. 11 



ENCODING METHOD IIOO 



IMAGE 



IIOI 



1102 

Jl 



BIT 
ALLOCATION 



i 



WAVELET 
TRANSFORM 



1103 

{ 



\ 



1105- 



ROI COEFFICIENT 
SCALING 



V 



QUANTIZER 



ENTROPY 
CODING 



1 



1 



1104 

I 



REGION 
SELECTION 



ROI COEFFICIENT 
IDENTIFICATION 



1106 



ROI COORDINATE 
DESCRIPTION 



1107 



1108 



1 



1109 



TRANSMITTER 



HEADER 


DATA 


BITS 


BITS 



9949412A1.I.> 



SUBSTITUTE SHEET (RULE 26) 



1 



WO 99/49412 



PCT/US98/19065 



28/34 



DECODING METHOD 1200 



Fig. 12 








HEADER 


DATA 




BITS 


BITS | 




\ 





I202 



i 



REGION 
COORDINATE 
DECODER 



ROl COEFFICIENT 
IDENTIFICATION 



T 
I20Z 



RECEIVER 



I20I 



I204 



ENTROPY 
DECODING 



r 



1205 



DE- QUANTIZATION 



1 



ROl COEFFICIENT 
DE- SCALING 



1206 



INVERSE 
WAVELET 
TRANSFORM 



1207 



RECONSTRUCTED 
IMAGE 



BNSDOCID: <WO 9949412A1_I_> 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/19065 



29/34 



IMAGE IS DIVIDED INTO BLOCKS 



(0) 


R 

(1) 


R 

(2) 




R c 


"L 


R° 


R \- 


R c 


R 


R° 


R ) 


RC 




R. 


-R 



Fig. 13 



ROl 



IMAGE 

EACH BLOCK IS ENCODED AS FOLLOWS • 



R c 




(i) 





ENCODER 
WITHOUT 
USING 
ROl 
FUNCTION 








TAG R c (i) 


ENCODED STREAM 









R° 
(k) 



ENCODER 
WITHOUT 
USING 

ROl 
FUNCTION 



TAG R° (k) 



ENCODED STREAM 



ENCODER 
WITH 
ROl 
FUNCTION 







STREAM 


STREAM i 




TAG R(j } 


FOR 


FOR 




ROl 


THE REST 



SUBSTITUTE SHEET (RULE 26) 



99494 12A1 I > 



WO 99/49412 



30/34 



PCT/US98/19065 



PROVIDE DIGITAL IMAGE 
DATA 



i 



SELECT REGION OF INTEREST 



130 



132 



PERFORM WAVELET TRANSFORM 
TO OBTAIN SUBBANDS OF 
COEFFICIENT 



IDENTIFY COEFFICIENTS 
CORRESPONDING TO REGION 
OF INTEREST 



i 



ORDER COEFFICIENTS 
INTO CATEGORIZE 



1 



134 



136 



138 



ENTROPY ENCODE COEFFICIENTS 
ACCORDING TO CATEGORY 



140 



Fig. 14 



BNSDOCID: <WO. . 9949412A1. 1. > 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



PCT/US98/19065 



31/34 



DETERMINE TOTAL NUMBER 
OF BITS TO BE USED TO ENCODE 
THE IMAGE 



142 



DETERMINE PRIORITIES FOR 
REGIONS OF THE IMAGE 



144 



ALLOCATE BITS TO ENCODE REGIONS 
BASED ON THE REGIONS PRIORITY SUCH 
THAT HIGH PRIORITY REGION CAN BE MORE 
ACCURATELY REPRODUCED THAN REGIONS 
OUTSIDE OF THE REGION OF INTEREST 



I46 } 



ENCODE IMAGE DATA 



ALLOCATE PORTION OF BIT STREAM 
TO BE USED TO TRANSMIT DATA 
CORRESPONDING TO HIGH PRIORITY REGIONS 



TRANSMIT BITSTREAM CONTAINING BITS CORRESPONDING 
TO BOTH HIGH AND LOW PRIORITY REGIONS 



152 



T 



WHEN ALL BITS CORRESPONDING TO HIGH PRIORITY 
REGIONS HAVE BEEN TRANSMITTED ALLOCATE ENTIRE 
BITSTREAM TO TRANSMITTING BITS CORRESPONDING 
TO LOW PRIORITY REGIONS 



r 

154 



Fig. 15 



SUBSTITUTE SHEET (RULE 26) 



? 



WO 99/49412 



PCT7US98/19065 



32/34 



600 



IMAGE DATA 



602- 



i 



ROi 
COORDINATES 



WAVELET TRANSFORMER 



604^ 
608- 



RATE 
A LLOCATOR 

618 



QUANTIZATION 
TABLE 



620 



ROI COEFFICIENT 
IDENTIFIER 



X 



606 



SUB BAND 
CLASSIFIER 



610 



CLASSIFICATION 
MAP 



i 



QUANTIZER 



612 



L 



ENTROPY 
ENCODER 



ROI 
PRIORITY 
INFORMATION 



614 



6/6 



ENCODED 
BITSTREAM 



622 



Fig. 16 



BNSDOC1D: <WO. 994941 2A1 I > 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 



33/34 



PCT/US98/I9065 



r 



QUANTIZATION 
TABLE 



ENCODED BIT 
STREAM 



622 



620 



L 



RATE 
DE~ ALLOCATOR 



T 

628 



ENTROPY 
DECODER 



L 



T 

624 



ROI 
PRIORITY 
INFORMATION 

\ 

6/6 



INVERSE 



CLASSIFICATION 
MAP 



610 





i 


SUBBAND 
DE-CLASSIFIER 



626 



630 



i 



INVERSE WAVELET 
TRANSFORMER 



630 



I 

IMAGE DATA 



632 



Fig. 17 



BNSDOCID: <WO 9949412A1. I_» 



SUBSTITUTE SHEET (RULE 26) 



WO 99/49412 PCT/US98/19065 



34 /34 



IMAGE 



ENCODER WITH 
ROI FUNCTION 



804 



TRANSMITTING SIDE 



TRANSMITTING 
DEVICE 



806 



RECEIVING 
DEVICE 



r 

820 



800 



RECEIVING 
DEVICE 



808 



DECODER WITH 
ROI FUNCTION 



TRANSMITTING 
DEVICE 



T 

818 



810 



DISPLAY 



812 



RECEIVING SIDE 



ROI SELECTOR 



814' 



ROI PRIORITY 
SELECTOR 



816 



J 



Fig. 18 



802 



BNSDOCID: <WO . 99494 12A1 1 > 



SUBSTITUTE SHEET (RULE 26) 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US98/19065 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC(6) :O06K 9/00 

US CL : Please See Extra Sheet 
According to International Patent Classification (IPC) or to both national classification and IPC 



FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 382/232, 233. 234, 235. 236, 238. 239. 240, 241, 242, 243. 244. 245, 246, 247, 248. 249. 250, 251. 252, 253 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



Y 
Y 



US 5,729,648 A (BOYCE et al) 17 March 1998, from col. 15, line 
48 to col. 16, line 68. 

US 4,811,210 A (MCAULAY) 07 March 1989, col. 5, lines 1-30. 

US 5,598,216 A (LEE) 28 January 1997, from col. 2, line 52 to 
col. 3, line 20. 



1-31 

1-31 
1-31 



j | Further documents are listed in the continuatiofl of Box C. | 1 See patent family annex. 



Special cm tog oo— of cited documents: 

document defining the general atate of the art which » not considered 
to be of particular relevance 

earlier document published on or after the international filing data 

document which may throw doubt* on priority claim(t) or which is 
cited to establish the publication date of another ortation or other 
aeon (as specified) 



later document published after the international filing date or priority 
date and not in conflict with the application but cited to understand 
the principle or theory under tying the invention 



document of particular relevance; 
considered novel or cannot be 
when the document is taken alone 



document referring to an oral disc Ice ure. use, exhibition or other 



document published prior to the international Tiling date but later than 
the priority date elaimed 



the claimed invention cannot be 
to involve an inventive step 



t of particular relevance; the claimed invention cannot be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, luch combination 
being obvious to a person skilled in the art 

document member of the same patent family 



Date of the actual completion of the international search 



23 NOVEMBER 1998 



Date of mailing of the international search report 



12 APR 1999 



Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D C. 20231 
Facsimile No. (703) 305-3230 



Authorized officer /"> 

LEO BOUDREAU 
Telephone No. (703) 308-7595 



p„c«, PnT/iRA/210 (second shoctX-mly 1992)* 

BNSDOCID: <WO. 99494 12A1. 1. > 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US98/19065 



A. CLASSIFICATION OF SUBJECT MATTER: 

US CL : 

382/232, 233, 234, 235. 236, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253 



Form PCT/ISA/2I0 (extra shcctXiuly 1992)* 

BNSDOCID: <WO 994941 2Al_l_> 



Page Blank (uspto) 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 



□ BLURRED OR ILLEGIBLE TEXT OR DRAWING, 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




ADED TEXT OR DRAWING 



This Page Blank (uspto) 



